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" ^identify  and  asslqn  priorities  to  the  experimental  questions  and  formulate 
an  overall  plan  for  their  I nvest I qat ion . (4)  determine  the  capability  and 
availability  of  the  research  equipment  available  for  carrying  out  the 
experimental  Investigations;  and  (5)  make  recommendations  for  carrying  out 
experiments  based  upon  considerations  of  priority  and  research  equipment 
capabl I Ity  and  aval labl I Ity,  v 

Due  to  the  largo  numberNjf^exper Iments  Indicated,  effort  was  concentrated 
on  the  development  and  test  tpe  performance  eaulvalence  method  for  determining 
trainino  simulator  characteristics  which  would  be  more  economical  of  time 
and  effort.  This  method  proposes  the  comparison  of  the  simulator  and  aircraft 
based  upon  the  measured  performance  of  the  pilot  In  terms  of  control  Inputs 
to  the  two  systems  being  compared.  Data  were  collected  In  an  Instrumented  T-37 
trainino  aircraft  as  the  first  step  In  carrying  pilot  performance  to  that  in 
the  ASUPT  simulation  system  at  Human  Resources  Laboratory  Flylnq  Training 
Division,  Viilliams  AFI3,  Due  to  administrative  and  schedellnq  constraints 
In  collecting  the  alrcrafl  data  left  time  in  the  contract  for  only  a breliminary 
analysis  of  those  data.  Equipment  and  scheduling  constraints  allowed  for 
collection  of  data  In  ASUPT  on  onlv  one  pilot  for  one  flight. 

Preliminary  analysis  of  the  aircraft  data  indicate  that  these  data  can 
be  used  profitably  in  the  derivation  of  measures  of  pilot  performance  for  test 
of  the  performance  equivalence  concent.  Since  no  tine  remained  In  the 
contract  to  carry  out  preliminary  analysis  of  the  ASUPT  data  no  projections 
can  be  name  as  fo  the  adequacy  of  "these  daia. 

An  overall  rrooram.  of  research  on  the  motion-vision  and  thel  Inter- 
action problem  was  outlined  based  upon  use  of  the  T-37  aircraft  data  as  a 
point  of  departure  in  configuring  ASUPT  as  a crlferlon  device.  The  carrying 
out  of  the  plan  if  successful  would  provide  ouleker  and  more  economical 
answers  to  both  simulator  characteri st I cs  and  trainino  methodoloay  questions 
provided  the  necessary  funds  and  personnel  support  could  be  brouaht  to  bear 
on  the  problem.  It  Is  recommended  that  the  program  be  pursued. 
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F I GURES 


1.0  INTRODUCTION 


Those  concerned  with  the  training  of  aircraft  pilots  have  been  working 
for  a long  while  toward  developina  efficient  and  effective  tralnlnq  methods. 
Their  work  has  been  directed  toward  methods  which  would  reduce  tralnlnq 
cost  and  time  and  to  increase  its’  quality  and  safety.  Recent  advances  In 
the  technolonv  for  nroduclnn  complex  training  devices  couoied  with  a 
greater  need  to  conserve  energy  resources  have  resulted  In  Increased 
Interest  in  obtaining  hard  data  for  answerina  the  auestlons  about  cost 
effective  simulator  deslon. 

The  investigations  reported  here  were  aimed  at  providing  answers  to 
a portion  of  these  questions.  Specifically,  they  were  designed  to  obtain 
Information  for  guidance  in  configuring  the  motion  and  external  visual 
characteristics  of  tralnlnq  simulators  to  be  used  within  the  several  levels 
of  pilot  training  of  the  United  States  Air  Force.  It  was  expected  that 
the  findings  would  be  aenera I i zafc I c to  training  in  other  tasks  which  renulre 
both  motor  skills  and  cognitive  information  processing.  While  the  specific 
characteristics  of  training  simulators  to  be  investigated  were  the  external 
visual  display,  simulator  motion  and  their  Interaction,  It  was  recognized 
that  thev  could  not  be  considered  in  isolation  from  other  Important 
variables  acting  In  the  training  situation.  Other  characteristics  of  the 
simulator  itself  are-  Important  and  interact  with  motion  and  vision,  A 
second  set  of  variables  Important  to  the  effectiveness  of  the  simulator 
are  those  subsumed  under  the  rubric  of  training  methodology.  This  iatter 
category  Is  concerned  with  such  Items  as  specificity  of  training 
objectives,  sequencing  of  training  objectives  and  tasks,  methods  of  feed- 
back, cognitive  Integral  Ion  and  a host  of  other  variables,  It  goes 
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without  saylna  that  these  variables  Interact  In  the  training  situation  and 
that  the  design  of  experiments  aimed  at  specifying  the  degree  or  level  of 
anv  one  for  effective  training  must  consider  the  many  levels  of  Its  inter- 
action with  the  other  variables.  Determining  all  of  the  effects  and 
combinations  of  effects  would  be  a lonq  and  tedious  process.  The  enormity 
of  the  number  of  experiments  needed  to  provide  answers  to  Questions  raised 
bv  these  Interactions  using  traditional  experimental  designs  and  methods 
caused  the  present  study  to  become  very  much  oriented  towaro  the  development 
of  more  efficient  experimental  and  evaluative  techniques.  These  techniques 
are  discussed  in  detail  In  later  sections. 

The  objective  of  obtalnlna  specific  data  for  recommendations  and 
nuidance  requ i red  stating  specific  ouestlons  and  defining  certain  procedures. 
These  are  stated  and  described  In  the  paragraphs  to  follow, 

1.1.  DESCRIPTION  OF  INDEPENDENT  VARIABLES 

A first  task  was  to  describe  and  auantlfy  the  Independent  research 
variables  within  the  areas  of  simulator  motion  and  visual  display. 

Although,  to  the  experienced  research  worker  this  point  would  seem  un- 
necessary of  belno  made  explicit,  by  far  the  greater  proportion  of 
research  on  these  variables,  particularly  that  of  simulator  motion,  has  been 
and  Is  being  carried  out  without  explicit  definition  of  the  variable. 

It  was  felt  to  be  an  Important  premise  that  the  research  variables  be 
defined  In  a manner  Independent  from  the  present  state-of-the-art  for 
Implementlno  them  in  simulators  and  along  a dimension  universally  meaningful 
to  the  simulator  deslqn  eneirieer.  He  could  then  Implement  the  findings 
In  the  most  efficient  way  that  he  could  devise  based  upon  the  state  of  the 
nnnlneerinq  art  at  the  time. 
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I .2  SCALING  OF  VARIABLES 


A second  and  corollary  task  to  be  addressed  In  dealing  with  the 
research  variables  was  the  means  for  establlshlna  the  perceptual  range 
and  the  scallna  Interval  of  the  variables.  The  answer  to  this  cuestlon 
bears  dlrectlv  unon  how  the  variables  might  be  used  In  other  experimental 
oaradlams.  More  imoortantly.  It  provides  direct  Information  to  those 
concerned  with  designing  and  procurinq  simulators  as  to  the  deoree  to 
which  variables  ordlnarllv  expressed  In  physical  "hardware  terms"  are 
perceived  and  acted  upon  by  the  operator  of  the  svstem, 

1.3  DEPENDENT  VARIABLE  ’’LASUREMENT 

The  third  major  area  of  concern  was  with  the  measures  and  methods  by 
which  the  resultant  or  dependent  effects  of  varying  the  experimental 
variables  was  to  be  assessed.  Corollary  to  the  specification  of  the 
measurement  and  scaling  of  the  Independent  variable  Is  the  effect  variations 
has  upon  the  outcome  of  Interest,  Traditional  methods  of  looking  at  the 
output  of  the  total  svstem,  while  appropriate  when  the  total  system  output 
is  the  onlv  Interest,  can  be  misleading  when  the  experimental  auestlon  Is 
concerned  with  the  effect  variation  In  the  system  has  upon  the  control 
behavior  of  the  operator.  Particular  emphasis  Is  placed  upon  this  point 
in  this  study  because  of  Its  importance  to  training  In  the  aircraft 
control  task  In  simulators  and  because  of  the  resistance  with  which  the 
concept  has  been  met  within  the  research  community.  Emphasis  In  this 
report  has  been  placed  upon  the  extreme  Importance  of  looking  at  the 
operator's  outnut  behavior  when  evaluating  or  comparing  systems  In  which 
the  operator  Is  being  trained  as  a closed  loon  compensatory  controller. 
Ignoring  measurement  of  the  controller's  output  Is  to  be  Ignorant 
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of  the  behavior  belnn  shaped  by  the  training  device. 

1.4  IDENTIFYING  EXPERIMENTAL  QUESTIONS 

The  fourth  area  of  concern  In  the  present  study  was  that  of  bringing 
tonether  and  assimilating  what  Is  known  about  the  effects  of  motion,  vision 
and  their  Interaction  In  the  simulator  situation;  making  explicit  certain 
experimental  nuestlons  to  be  answered;  and  assigning  priorities  to  those 
ouestlons.  The  criteria  for  priorities  can  change  as  conditions  and 
requirements  within  the  Air  Force  and  its  working  environment  change. 

For  example,  the  necessity  for  maktnn  decisions  about  simulators  for 
specific  systems  under  development  and  procurement  may  bring  added 
pressure  for  data  upon  which  to  base  decisions  specific  to  those  systems. 

At  the  same  time  such  particular  and  specific  questions  must  be  welohed 
aqalnst  the  necessity  for  defining  basic  ouestlons  and  obtalnlna  data 
qenera I I zab 1 e across  many  systems.  Often  this  latter  research  Is  put 
aside  In  favor  of  answering  a specific  auestlon  at  a given  time  or  utilizing 
a specific  piece  of  experimental  apparatus, 

1.5  RESEARCH  EQUIPMENT,  CAPABILITY  AND  AVAILABILITY 

A further  Important  factor  In  setting  priorities  and  obtaining 
answers  is  the  availability  and  capability  of  research  equipment  and 
resources  for  obtaining  empirical  data.  It  should  be  axiomatic  In 
expenimental  research  that  the  experimental  auestlon  should  be  stated 
specifically;  experimental  variables  defined  explicitly;  the  measurement 
technl aues del ineated;  and  finally,  the  experimental  apparatus  designed 
and  procured  for  Investigation  of  the  stated  ouestlons.  This  utopia  Is 
rarelv  obtainable  even  for  ratTT^F'TtmD I e nuestlons  dealt  with  In  a 
■laboratory  situation.  It  Is  certainly  less  likely  to  be  realized  In 
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working  with  the  complex  problems  being  dealt  with  In  pilot  training  with 
Its  reoulrement  for  complex  advanced  state-of-the-art  and  expensive 
research  enulpment.  The  eaulpment  designed  to  be  the  most  complete  tor 
studying  the  Questions  addressed  In  this  study  was  felt  to  be  the  ASuPT 
simulator  located  at  the  Air  Porce  Human  Resources  Laboratory,  Flylna 
Trainino  Division,  AFSC,  Williams  Air  Force  Base.  It  was,  therefore,  a 
part  of  Intent  of  this  project  to  determine  the  capabilities  of  this 
research  enulpment  and  to  use  this  capability  as  one  of  the  factors  In 
dptnr1- i n i nc  research  priorities, 

I.  r;  !jMf.[)  PC  SEARCH 

:<asm  upon  the  considerations  and  findings  In  the  above  five 
'a'a-rarhs  it  was  the  purpose  of  this  study  to  combine  this  Information 
v.-iln  that  of  a parallel  study  of  training  research  priorities  based  upon 
+ne  concensus  of  experts  In  the  field  and  to  evolve  a short  and  long  term 
research  program  specific  to  obtaining  answers  to  the  Questions  of  vision 
ano  notion  simulation  in  pilot  training  simulators.  It  was  also  to  be  the 
resnonsibi I Ity  of  personnel  involved  In  this  project  to  assist  In  the 
conduct  of  critical  experiments  in  the  high  prlorty  areas,  analysis  of 
the  data  and  i ntororetat Ion  and  application  of  the  results. 

It  Is  the  purpose  of  this  report  to  summarize  the  proqress  and  the 
problems  in  attempting  to  carry  out  the  activities  listed  In  the  above 
ParaGraphs,  as  guidance  both  for  the  design  of  simulators  and  for  future 
research. 
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2.0  DESCRIPTION  OF  EXPERIMENTAL  VARIABLES 
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2.1  MOTION  VARIABLES 

It  was  indicated  in  the  earlier  section  that  one  of  the  important 
items  to  be  accomplished  In  the  project  was  the  definition  of  the 
independent  experimental  variables.  It  was  also  Indicated  that  specific 
definition  of  the  motion  variable  is  reouired  in  order  to  manipulate  It 
as  an  Independent  variable.  Just  as  importantly,  it  should  be  defined 
such  that  it  is  meaningful  and  useful  to  the  simulator  design  engineer. 

Motion,  as  it  may  be  sensed  by  the  operator  and  Influence  his  control 
hehavlor,  has  been  considered  In  this  study  as  being  of  two  major  types, 
These  are  (I)  external  forcing  functions  of  which  there  are  two  sub- 
cateoories,  and  (2)  movement  as  a result  of  control  movement, 

2.1.1  External  Forcing  Functions 

The  external  forcing  function  to  which  the  system  (aircraft)  reacts 
and  to  which  the  operator  In  turn  responds  may  be  imparted  to  the  system 
through  changes  in  air  density  or  movement  (turbu lence) , or  they  may  be 
inputs  to  the  system,  either  external  or  Internal,  which  may  be  termed 
abnormal  or  catastrophic  Inputs.  Each  of  these  two  types  is  elaborated  In 
the  paragraphs  below. 

The  most  important  type  of  aircraft  motion  which  must  be  Identified 
and  desci ibed  is  that  of  the  response  of  the  aircraft  to  the  normal  and 
continuously  actino  inputs  to  It  from  the  turbulence  to  which  It  is 
subjected.  In  examining  this  source  of  motion,  Interest  Is  upon  Its 
effect  upon  pilot's  ability  to  carry  out  the  compensatory  control 
activities  reouired  to  direct  his  aircraft  through  three-dimensional  space. 


6 


In  order  to  direct  his  aircraft  through  this  space  he  must  exert  direct 
control  over  the  pitch,  roll,  and  yaw  axes  of  his  aircraft,  External 
forces  which  tend  to  force  his  aircraft  from  the  attitude  which  he  Is 
attempting  to  maintain  must  be  sensed  by  him  and  corrections  made. 

The  major  dls+urbances  to  his  aircraft  from  these  external  forcing 
^unctions  act  to  disturb  the  system  In  pitch,  roll  and  along  the  z or 
heave  axis.  The  operator's  sensing  of  these  disturbances  can  serve  as 
cues  to  him  In  exercising  control  over  the  oltch,  roll  and  the  yaw  axes 
and  thus  null  the  error.  It  Is,  therefore,  important  that  the  response 
characteristics  of  the  system  being  controlled,  as  It  Is  disturbed  In 
its  operating  environment,  be  Identified  and  described  In  a quantitative 
fashion  so  that  its  importance  to  operator  control  may  be  determined. 

The  second  type  of  forcing  function  Is  similar  in  Its  effect  to  a 
turbulence  Input  put  can  be  differentiated  in  that  It  Is  ordinarily  not 
continuous  and  may  regulre  a very  rapid  but  rarely  occurlng  response  on 
the  part  of  the  controller.  An  often  cited  example  of  this  is  the  sudden 
loss  of  power  in  an  outboard  engine  on  a transport  type  aircraft.  In 
such  a situation  the  sensed  motion  may  provide  the  operator  with  the 
information  necessary  for  him  to  make  corrective  Inputs  Into  the  system, 
2,1.2  Control  Input  Motion 

The  second  major  type  of  motion  Is  that  which  the  system  makes  In 
response  to  inpufs  into  the  controls,  The  Immediate  feedback  to  the 
operator  after  making  a control  Input  is  the  motion  of  the  system  about 
the  pitch,  roil  and  yaw  axes  and  In  response  to  changes  In  power  or 
■thrust.  An  Important  source  of  movement  in  some  systems  is  Its  response 
to  the  release  of  stores  of  various  kinds.  This  response  can  be 
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cons i drsred  an  external  forclnn  -function  leadlnn  to  a disturbance  In  the 
system  to  be  sensed  and  controlled  by  the  opersTor. 

It  was  pointed  out  In  Section  I that  a highly  complex  Interaction  of 
variables  must  be  considered  In  studylnn  the  characteristics  of  simulators 
o*  aircraft.  With  respect  to  control  movement  motion  It  is  believed 
necossarv  to  point  out  at  this  time  the  Importance  of  the  kinesthetic 
feedback  from  the  controls  and  its  relationship  to  notion  and  visual  feed- 
back. In  trainers  and  simulators  of  aircraft.  In  such  system?  when  the 
operator  Is  attempt  I no  to  perform  a compensatorv  tracklno  task  and  to 
null  errors  as  thev  occur.  3 deviation  from  the  standard  may  come  about 
.as  a function  o*  el+her  a control  input  or  as  a result  of  an  externally 
anplled  forcinn  function.  It  Is  Important  to  the  operator  in  his  performance 
of  precise  control  that  he  be  able  to  separate  those  deviations  occurring 
from  external  forcino  functions  and  those  occurrlnq  as  a result  of  control 
movement  and  that  he  be  able  to  do  this  on  a proportional  basis.  That 
Is  to  sav  that  it  is  necessary  for  him  to  learn,  based  upon  kinesthetic 
feedback  from  his  control,  what  deviation  to  expect  for  a given  amount  of 
control  input  so  that  if  simultaneous  +o  his  control  Input  an  external 
forcinn  function  acts  either  to  amplify  or  attentuate  the  deviation  he 
can  identify  It  and  adapt  his  control  accordlnr.lv, 

2.1.3  "otion  as  an  tx  :er imrnta!  Variable  In  the  Simulator 

One  of  the  major  deterents  to  obtaining  objective  and  genera  I I zab ! e 
data  with  respect  to  the  relationship  between  motion  and  performance  has 
Peon  the  lack  of  specificity  In  the  definition  of  motion  as  an  exoerl- 


mental  variable.  Often  no  hint  of  the  characteristics  of  the  motion 
platform  under  Invest inat Ion  Is  given,  Unless  simulator  platform  motion 
is  defined  In  terms  and  with  a metric  which  allows  Its  precise 
Implementation  as  an  experimental  variable  or  as  a design  parameter  the 
euestlon  of  the  effect  of  motion  will  remain  In  limbo. 

The  basic  problem  in  defining  simulator  motion  Is  to  establish  a 
means  of  measurinn  the  physical  fact  of  the  motion  of  a platform  sc  that 
the  amount  of  motion  can  he  varied  In  precise  measurable  ways  and  to  do 
this  In  a wav  In  which  Is  meaninoful  and  useful  to  the  simulator  design 
enp I neer. 

The  physical  reality  of  the  chanoe  In  position  of  a mass  such  as  a 
simulator  motion  olatform  may  be  modeled  and  described  mathematically 
throunh  description  of  that  change  In  position  and  the  derivatives  of  that 
change.  The  fidelity  problem  In  simulation  Is  one,  first  of  all,  of 
determining  whether  the  changes  of  position  of  the  platform  and  the 
derivatives  of  those  chances  are  like  those  exhibited  by  the  vehicle  belno 
simulated,  l.e.,  the  aircraft.  Obviously  the  simulator  does  not  duplicate 
the  aircraft  with  respect  to  the  mannltude  of  the  changes  In  position. 

It  may  be  made  to  duplicate  the  aircraft  with  respect  to  certain  of 
derivatives  provided  that  the  results  of  the  Integration  of  these 
derivatives  does  not  exceed  some  final  value,  We  can  do  this  by  setting 
Into  motion  a counter  <orce  or  motion  In  the  opposite  direction  come  to 
be  termed  "washout”. 

The  capability  of  a simulator  motion  platform  for  transmitting 
movement  to  the  cockpit  and  to  the  pilot  may  be  described  Quantitatively 
In  terms  of  Its  ability  to  reproduce  certain  rates,  accelerations,  and 
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rates  of  onset  of  acceleration  exhibited  by  the  aircraft  and  perceived  by 
the  pilot  as  Information  pertinent  to  his  control  of  the  svstem.  Due  to 
the  limited  positional  change  capability  of  the  simulator,  however,  the 
washback  feature  must  be  Incorporated  during  which  the  same  physical 
asoocts  of  motion,  t.e.,  rates,  accelerations  and  rates  of  onset  of 
accelerat ions  are  Involved.  In  this  case  their  magnitudes  must  be  kept 
below  the  percentable  limits  of  the  pilot. 

The  response  of  the  simulator  platform  may  be  described  as  Its 
ability  to  respond  to  a given  Input  or  Impulse  to  the  platform  and  this 
response  could  be  described  In  terms  of  the  way  It  moves  over  time. 

Thus,  If  the  platform  were  driven  by  a sinusoidal  Input  with  varying 
frequency,  the  output  or  the  way  In  which  the  platform  followed  this 
Input  could  be  plotted  as  a function  of  time,  i.e,,  It  could  be  described 
in  the  time  domain.  This  method  of  looking  at  the  response  of  the  plat- 
form perhaps  makes  it  more  cl  earl v understood  how  a platform,  due  to  Its 
particular  characterl st I cs,  may  not  follow  the  Input  precisely.  In 
part  cular,  as  the  Input  (the  sinusoidal  driving  function)  Increases  In 
freouency  the  platform  may  fal I behind  the  Input  both  In  terms  of  the 
amplitude  and  the  froouency  of  Its  response,  l.e,,  It  will  begin  to  lag 
behind  the  Innut.  These  at+rlbutes  of  the  response  of  the  platform  to 
a sinusoidal  Innut  may  be  described  In  terms  of  Its  freouency  response 
and  Its  phase  lac,  t.e..  It  may  be  described  In  the  frenuencv  domain. 

An  Important  consl deration  In  the  description  of  the  response  of  the 
motion  platform  In  the  freouency  domain  Is  that  of  the  driving  or  forcing 
function.  A standard  method  for  determining  the  freouency  response  of 
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the  platform  night  be  as  Indicated  above,  l.e,,  that  of  driving  the  plat- 
monon  piaTtorm  couia  ne  conriaurea  to  respona  line  xne  open-iooo  response 


of  the  aircraft  with  resnect  to  rates  of  onset  of  accelerations 
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accelerations,  rates  and  so  forth.  However,  with  the  simulator  platform 


an  Deposing  movement  must  be  Initiated  before  Impractical  positional' 
changes  have  taken  place.  Thus,  If  these  opposing  forces  ace  initiated 
before  large  rates  or  positions  have  occurred  the  less  complex  the 
platform  Is  likely  to  be.  The  question  becomes  one  of  whether  or  not 
the  platform  can  be  made  to  provide  the  pilot  with  useable  motion  cues 
to  his  control  while  It  Is  limited  to  low  positional  and  rate  changes, 

A note  of  caution  must  also  be  sounded  with  respect  to  the  determination 
of  the  response  frequency  of  the  platform  through  use  of  an  open-loop 
driving  sinusoidal  function.  The  open-loop  test  Is  a straight  forward 
wav  of  determining  the  transfer  function  of  the  device  being  examined, 
l.e,,  Its  ability  to  follow  the  Input  values  and  a description  of  how  It 
follows.  However,  If  the  system  Is  closed-loop  system,  l.e,,  an  operator 
(pilot)  is  placed  within  the  system,  he  senses  the  output  and  acts  upon 
It  In  a particular  way.  We  may  find  that  he  Is  adding  something  to  his 
control  which  makes  the  system  respond  faster  than  It  would  appear  to 
respond  In  solely  open- loop  fashion.  That  Is  to  say,  that  the  pilot  may 
anticipate  or  act  upon  Information  received  by  him  and  make  Inputs  Into 
the  system  which  provide  lead  or  anticipatory  Information,  The  result 
would  be  to  drive  the  system  faster  than  It  would  appear  to  respond  under 
the  open-loop  situation.  Thus,  the  frequency  response  required  In  a 
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simulator  platform  might  very  well  be  higher  than  would  appear  to  be 
necessary  with  an  open-loop  test  comparison  of  the  platform  with  the 
a I rcraft. 

In  any  case,  the  freouency  response  method  of  examining  the  fidelity 
of  motion  characteristics  of  a simulator  appears  to  be  very  feasible  and 
one  which  Is  useful  to  the  simulator  design  engineer.  What  Is  Involved 
Is  the  determination  of  the  upper  breakpoint  freauency  response  of  the 
simulator  platform  which  determines  how  fast  It  will  respond  and  the 
lower  breakpoint  freouency  which  determines  In  effect  how  fast  It  will 
washout  or  return.  Past  research  would  Indicate  that  the  upper  break- 
point freauencv  of  the  platform  should  be,  at  least,  2,5  hz  and  the  lower 
breakpoint  freouency  could  be  as  hinh  as  ,5  hz.  However,  these  values 
are  not  well  established  and  a useful  research  motion  platform  should  be 
capable  of  responding  up  to  approximately  3.5  hz  at  the  upper  breakpoint. 

In  summary,  the  description  of  the  fidelity  of  simulation  variable 
for  anv  dimension  o*  motion  Is  recommended  to  be  the  freouency  response 
of  the  platform  to  6 standard  driving  Input,  The  range  of  this  freauency 
Is  recommended  to  be  from  0 to  3,5  hz  for  a research  simulator. 

2.2  VISUAL  DISPLAY  VARIABLES 
2.2.1  Visual  System  Designs 

All  visual  simulation  systems  reoulre  that  some  method  be  devised  to 
generate  an  Imaoe  and  some  method  also  be  devised  to  display  that  Imaqe, 
Taylor  et  a I . * have  listed  four  basic  approaches  to  visual  simulation,  all 
of  which  are  extant  In  one  form  or  another  In  simulators  In  use  today. 
These  are  (a)  the  model  system  approach,  (b)  the  film  projection  system 
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approach,  (c)  a system  Involving  transparency  reconstruction,  and  <d)  the 
electronic  Image  generation  system  approach. 

The  model  system  Involves  the  construction  of  a scale  model  of  a 
scene  to  be  simulated,  A TV  camera  or  other  optical  pickup  device  Is 
usually  o laced  on  gimbals  or  servo  mechanisms  located  over  the  model  and 
the  optical  plck-ur  system  then  scans  the  model  with  the  pilot  operating 
the  aircraft  controls  In  the  simulator  according  to  the  picture  displayed. 

Film  projection  systems  provide  two-dimensional  representations  of  a 
three-dimensional  scene  via  motion  picture  film  or  film  strip. 

In  transparency  reconstruction,  accordlna  to  Taylor  et  al.,  a constant 
hioh  Intensity  beam  of  electrons  Is  swept  In  time  across  the  low  persistence 
screen  of  a flying  spot  scanner  In  a specified  raster  pattern.  The  raster 
Is  focused  on  rne  transparency  which  modulates  an  Intensity  the  light 
flux  passing  through  It.  This  time  varying  flux  is  detected  by  light 
sensitive  photo  multipliers  which  In  turn  aenerate  voltages  proportional 
to  the  Incident  light  flux. 

Finally,  these  authors  list  electronic  Image  generation.  This 
Involves  the  generation  of  a visual  Image  by  a digital  computer  system 
which  has  stored  the  aata  describing  the  visual  environment  In  the 
computer  memory  and  solves,  In  real-time,  a mapping  transformation 
function  which  defines  the  environment  onto  the  display  or  Image  plane. 

Two  things  are  to  be  considered  concerning  the  four  qeneral  methods 
for  generating  visual  displays.  First,  any  attempt  to  specify  and 
ouan+lfy  Independent  variables  for  evaluation  and/or  generation  of  a 
visual  simulation  dlspi  iy  should  be  Independent  of  the  techntoue  which 
Is  called  for  In  the  development  of  the  display.  That  Is,  specification 
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of  the  Indeoendent  variables  should  not  be  contingent  upon  whether  the 
Images  are  deveiooed  from  a model  or  from  a transparency,  etc.  Second, 
the  Image,  whether  It  is  oenerated  bv  any  or  all  of  these  four  listed  above 
has  certain  characteristics  which  have  been  addressed  by  researchers  In 
times  past.  It  Is  to  this  second  point  that  comments  are  now  directed, 

2.2.2  Visual  display  Character! sties 

Researchers  have  postulated  several  approaches  to  the  evaluation  of  a 
visual  disnlav.  In  the  evaluation  of  these  displays  there  seems  to  be 
some  consensus  with  resnect  to  certain  criteria  which  an  Image  should 

attain  In  order  to  be  acceptable.  A convenient  dichotomy  has  been  proposed 

2 

by  Rouoelot  , who  sunnested  that  Images  should  exhibit  desired  character- 
istics of  duality  and  content.  Qualitative  characteristics  he  listed  were 
brightness,  resolution,  color,  focus,  and  contrast,  he  listed  under  the 
heading  of  content:  wide  field  ot  view,  realism  (detail),  unrestricted 

environment  dynamics  (e,r,,  controllable  moving  objects  In  the  environment) 
and  unrestricted  environment  coverage.  Other  researchers  (e.q.,  Rosendahl"5, 
have  Indicated  that  all  Imaoes  should  meet  some  criterion  ot  resolution. 

He  has  sugoested  that  In  addition  to  resolution,  sharpness  of  the  optical 
Imaoe  Is  Imnortant.  He  went  further  to  sumest  that  acutance,  which  Is 
the  normalized  mean  sauare  average  of  the  slope  of  an  edge  trace,  Is  a 
much  better  expression  of  Imago  sharpness.  Other  researchers  and/or 

documents  (see  for  example  the  ATA  Visual  Simulation  Sub-Committee 

4 

recommendations  for  visual  specif IcaMons  for  today's  simulator  ) also 
sneak  of  such  characteristics  as  brightness,  resolution,  contrast, 
distortion,  perspective,  etc. 
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The  use  of  Rouoelot's  dichotomy  Is  a convenient  way  to  demonstrate 
that  there  are  certain  characteristics  of  a visual  dlsnlay  which  have  to 
do  with  the  cuallty  of  that  disrlav  while  there  are  certain  othc  - 
character  I st i cs  of  this  display  which  Identify  the  Imaqe  or  dlsnlay  content. 
Further  research  Into  the  backoround  of  this  dichotomy  (see  for  example 
Tavlor  et  al.)  Indicates  that  Inane  duality  characteristics  such  as 
brinhtness,  resolution,  sharpness,  contrast,  distortion  and  perspective 
are  reasonably  well  ouantifled  and  certain  recommendations  can  he  made 
to  the  ennlneer  with  respect  to  tan  amount  of,  say,  distortion  which  should 
be  permissible  in  a oiven  visual  display.  It  Is  the  approach  here  that 
there  are  optimum  eua||tative  character i st i cs  of  a visual  display  such 
that  one  would  expect  optimum  perspective,  little  or  no  distortion, 

Contrast  within  3 specified  rynpe,  a certain  sharpness  I ev« I on  the  nart 
of  the  visual  disnlav  and  a display  that  meets  some  criterion  for 
resolution  and  brlohtness.  Therefore,  the  major  emphasis  In  this  report 
has  hden  on  those  variables  which  could  be  subsumed  under  a headlna  of 
Inace  Content. 

A second  reason  for  pursuing  variables  under  the  head  I nq  of  Imane 
Con ten f could  be  explained  on  the  basis  of  the  fact  that  these  variables 
are  no f covered  to  any  great  extent  In  the  literature.  Thus,  It  Is  diffi- 
cult to  provide  the  des'pn  enpineer  with  data  in  quantitative  form  which 
accurately  and  adenuately  defines  the  content  which  3 visual  Image  should 
possess  for  a oiven  simulation  requirement. 

It  Is  pronosori  that  Those  variables  to  he  considered  Initially  under 
the  headlno  Imane  Content  aro:  detail,  complexity  (number  of  objects  per 

un i 1 area),  teal  ism,  and  texture. 


2.2.3  I mane  Content  Ouantl f ! cat  Ion 

There  are  two  approaches  to  the  auantl f Icatton  of  the  variables  listed 
under  the  headlna  Imaae  Content.  It  Is  possible  to  specify  a given  content 
variable  and  propose  the  metric  to  be  used  to  vary  that  variable  along  some 
continuum.  It  Is  also  possible  that  one  might  experiment,  as  It  were,  with 
the  variable  In  terms  of  qualitative  differences  In  the  variable  to  make 
some  determination  as  to  whether  this  variable  merits  further  study.  That 
Is,  some  variables  may  be  added,  deleted,  or  chanqed  as  a function  of  the 
exnerlmental  conditions  surroundlnn  them.  For  that  reason,  some  of  the 
varkb.f.j  proposed  have  more  precise  definitions  and  a more  precise  metric 
than  other  variables  and  exploratory  studies  are  needed  before  a final 
judnment  Is  made  to  Include  or  exclude  these  variables. 

The  conventional  approach  to  the  specification  of  these  Imaae  content 
variables  would  dictate  that  one  need  merely  count  the  number  of  objects 
within  a given  visual  scene  In  order  to  describe  the  content  of  that  Image, 
However,  this  particular  approach  Is  lacking  In  that  It  Is  difficult  to 
establish  a rule  or  model  on  which  to  base  the  number  of  olven  objects  one 
might  Include  In  a given  scene.  That  Is  to  say,  one  needs  a rule  for 
Inclusion  or  exclusion  of  a given  object  or  type  of  object  within  the 
scene.  In  addition,  detail  and  Image  content  are  confounded  In  that  one 
miaht  establish  the  relative  degree  of  Imago  content  on  some  scale  from 
hlph  to  low  and  on  the  basis  of  the  amount  of  deta! I Judged  to  be  present 
in  a visual  scene.  Thus,  a scene  rated  as  having  a high  amount  of  detail 
would  be  judoed  as  possesslnn  a high  level  of  Imaae  content.  On  the  other 
hand,  a scene  with  very  ll+tle  detail  would  be  judoed  to  be  very  low  In 
Image  content.  This  mav  be  one  case  of  a more  neneral  condition  but  It 
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does  not  lend  itself  to  quantification  or  description  as  an  Independent 

variable  due  to  the  fact  that  one  has  no  convenient  metric  for  the 

Inclusion  or  exclusion  of  a qiven  object  or  number  of  objects  In  the 

scene.  It  Is  also  the  case  that  a scene  with  very  low  detail  could  be 

judaed  as  possess inq  a high  level  of  content.  That  Is  to  sav,  one  might 

see  a nreater  number  of  objects  In  a given  field  with  a low  level  of 

detail  per  object.  Because  of  this  a paradigm  Is  developed  In  the 

followlnq  discussion  that  dlsnenses  with  the  notion  of  Image  content  in 

the  sense  of  the  number  of  objects  reaulred  In  a given  scene  and  approaches 

the  category  of  Imaae  content  on  the  basis  of  the  cues  reaulred  for  the 

pilot  to  accomplish  one  of  the  several  categories  of  maneuvers  reaulred  of 

him.  Figure  2,1  Is  a block  diagram  of  the  paradigm  which  shows  two  stages 

based  on  the  type  of  Information  reoulrea  to  accomplish  a particular  task. 

These  stages  are  predicated  on  the  notion  that  the  pilot  reaulres  two 

kinds  of  visual  Information  In  order  to  accomplish  his  task;  position 

Information  and  rate  Information.  The  paradigm  Itself  was  developed  out 

5 

of  a model  for  visual  discrimination  (Thlelges  & Matheny  ) which  analyzes 
the  visual  scene  external  to  the  aircraft  In  terms  of  perspective  geometry. 
That  is,  a picture  plane  is  chosen  to  lie  coincident  with  the  aircraft 
windscreen  and  perpendicular  to  the  eye  line  of  regard  so  that  an  external 
referent  is  projected  on  this  plane.  Simultaneously  an  Internal  referent 
exists  on  the  piano.  Thus,  the  external  and  Internal  referents  lie  In 
juxtaposition  one  to  the  other  and  the  essential  task  of  the  pilot  Is  to 
detect  perturbat I ons  of  the  vehicle  on  the  basis  of  the  relationship  or 
change  In  the  relationship  of  the  Internal  and  external  referents  on  this 
picture  plane. 


17 


ii 


i 

: 

i 


Stage 

Requ 1 red 
I n 

Imaqe 

Task  Used 
For 

1 nformat 1 on 
Obta 1 ned 

System  of 

Detection  & 

Posit  Ion 

1 . 

1 nterna 1 & Externa  1 
Referents 

identl f 1 cat  Ion 
of 

Externa  1 
Referent 

Surface  & 

1 dent  I f 1 cation 

Position 

2, 

Sky 

A 

& 

Texture 

T rack  I ng 

Rate  Info, 

Figure  2,1  Prooosed  Two-Stage  Visual  Cue  Paradigm, 

Referrinn  +o  Figure  2.1,  the  first  stage  Indicates  that  If  one  Is 
given  a system  of  referents  (in  this  case  the  internal  and  externa  I 
referents  alluded  to  above)  one  may  obtain  position  information  from  the 
juxtaposition  of  these  referents  and  this  juxtaposition  will  serve  then 
for  the  detection  and  Identification  of  an  external  referent.  Alternately, 
the  first  staae  considers  the  task  of  the  pilot  to  be  detection  and 
Identification  of  some  external  referent  which  Is  usually  in  the  form  of 
objects  or  scenes  or  targets  within  the  field  of  view.  This  paradigm  Is 
independent  of  the  maneuver  renul red  of  the  operator,  l.e,,  whether  It  Is 
to  be  take-off  and  landlno,  formation  flight,  etc.  A olven  task  might  be 
to  Identify  an  external  object  which  would  be  designated  the  external  referent 
(EP)  and  which  the  pilot  must  identify  and  act  upon.  Identification  of  the 
external  referent  requires  position  Information  with  respect  to  that  external 
referent.  For  example,  if  the  pilot  were  renul red  to  join  uo  with  another 
aircraft  for  formation  flight  the  I R In  the  given  field  would  he  the 
aircraft  with  which  the  pilot  must  join.  Thus,  a system  of  external 
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referents  is  required  In  order  to  provide  position  Information  to  accomplish 
the  tasks  of  detection  and  Identification  of  an  external  referent  to  which 
at  some  later  time  some  mean  I no  will  be  attached. 

The  second  stage  discusses  an  additional  tasx  in  addition  to 
Identification;  namely,  tracking.  In  order  to  track  a referent  external 
to  an  aircraft  the  pilot  Is  renuirod  to  have  some  type  of  rate  data  to 
plve  him  lead  or  oredictiye  Information.  In  this  case  rate  data  could 
he  provided  bv  surface  or  skv  texture.  These  textures  can  be  Identified 
or  defined  as  the  textures  of  either  the  ground  surface  or,  In  the  case 
of  joining  un  with  an  additional  aircraft  In  formation  flight,  the  air- 
craft Itself  or  other  objects  In  the  immediate  vicinity  of  the  pilot. 

To  recapitulate,  the  paradigm  of  Image  content  for  a glyen  visual 
disolav  has  two  parts  wnich  are  divided  on  the  basis  c*  wnpther  the 
individual  must  tderitlfv  or  whether  he  must  track  a given  object  within 
the  scene,  If  detection  and  Identification  Is  the  task  then  the  subject 
needs  positioning  Information  which  Is  provided  by  objects  In  the  field. 

If  the  Individual  must  then  track  these  objects  It  Is  necessary  to  have 
rate  information  which  can  be  provided  by  surface  or  sky  texture.  It 
should  be  noted  at  this  point  that  for  an  Individual  To  accomplish  a 
given  visual  task  he  will  be  both  'dentlfylng  and  tracking  an  external 
referent.  Therefore,  his  task  is  actually  one  of  time  sharing  between 
Identification  and  tracking  In  a o I ven  maneuver  and  this  maneuver 
dictates  whether  he  i s to  spend  most  of  his  time  Identifying  or  most  of 
his  time  tracking , 

Finally,  In  Its  simplest  form  the  model  states  that  the  two  stages 
provide  Information  for  detection  and  Identification  through  a system  of 
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internal  and  external  referents  and  provides  for  detection  of  rates 
through  surface  or  skv  texture.  Although  the  model  Is  Incomplete  It  Is 
convenient  as  a vehicle  to  test  the  first  list  of  proposed  variables, 
namely  detail,  complexily,  realism  and  texture.  Further,  It  is  necessary 
to  test  these  variables,  In  part  because  It  Is  doubtful  that  one  would 
find  visual  simulation  systems  acceptable  which  do  not  have  a modicum  of 
face  validity.  It  Is  perhans  necessary  that  displays  show  or  project 
images  of  what  the  pilot  expects  to  see  In  the  so-called  real  world 
situation,  even  If  these  add  nothing  of  value  to  the  training  situation. 
2.2.4  Image  Content  Variables 

Complexity  of  content  Is  operationally  defined  as  fhe  number  of 
objects  per  unit  area.  Complexity,  In  the  qeneral  sense,  simply  means 
how  cluttered  the  Image  Is.  It  is  pertinent  to  our  general  model  In  that 
one  can  vary  the  level  of  complexity  as  defined  above  and  therepy  test 
certain  tenents  of  the  model  with  respect  to  embedding  ex+emal  referents 
In  the  Image,  placing  the  requirement  on  the  p i lot-obseryer  to  detect  and 
iden+!fy  the  external  referent  and  then  act  upon  It,  As  such,  complexity 
is  one  of  those  variables  which  was  suggested  earlier  as  one  which  should 
first  take  a categorical  or  qualitative  form  for  further  exploration. 

That  is,  one  should  not  expect  at  the  outset  to  be  able  to  quantify  the 
variable  complexity  without  f , rst  investigating  complexity  on  some  molar 
level  to  make  determinations  as  to  what  particular  characteristics  or 
parts  of  +h!s  variable  complexity  are  amenable  +o  ouant I f Icat Ion.  Simply 
stated,  a scene  with  a standard  unit  area  which  has  relatively  few  objects 
discernible  within  that  area  would  be  considered  a scene  with  a low  level 
of  complexity.  In  contrast  a scene  with  a unit  area  which  has  a large 
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number  of  objects  discernible  within  this  area  would  be  judqed  a scene  a 
hloh  level  of  complexity.  For  purposes  of  Initial  study,  therefore,  It  Is  1 

preferable  to  consider  complexity  in  terms  of  categories  of  high  and  low  ' 

levels  (or  simple  vs.  complex)  with  the  relative  numbers  o*  objects  with- 
in a olven  unit  area  operationally  deflnlnp  these  categories,  The  out- 
come of  experimentation  with  this  variable  Is  such  that  the  variable  would 
Drove  to  be  either  a meaningless  term  or  one  which  could  be  explored  further 
with  ultimate  cuant i f i cat  I on  of  the  objects  In  terms  perhaps  of  optimum, 
minimum,  and  maximum  requirements  for  the  number  of  objects  within  a given 
unit  area,  not  only  on  the  basis  of  what  Is  necessary  and  sufficient  but 
what  Is  iudoed  to  be  acceptable  to  the  observer. 

Coincident  with  the  construct  complexity  one  needs  also  to  consider 
the  notion  of  realism.  Realism,  per  se,  of  a given  image  !s  actually  a 
meaningless  term.  In  one  sense  it  nloht  be  said  that  there  Is  no  such 
thing  as  a realistic  scone.  This  is,  at  least,  partly  true  from  an 
objective  point  of  view  owing  primarily  to  the  fact  that  realism  Is  a 
two-pronoed  term.  It  depends  partly  on  the  content  or  complexity  of 
the  given  scene  and  part  I y on  the  perceptual  characteristics  of  the 
observer  in  terms  of  his  perceptual  set,  central  nervous  svstem  mechanisms, 
and  peripheral  receptors.  All  o*  this  Is  1o  say  that  the  observer  brlnas 
with  him  to  the  visual  scene  certain  expectancies  as  to  what  the  scene 
should  tie  like  and  these  expectancies  dictate  what  Is  figure  and  what  Is 
ground.  Also,  these  expectancies  will  serve  somewhat  to  reject  or  accept 
a niven  sc  ne  which  defines  it  as  more  or  less  realistic  and  these  are  as 
much  Individual  idiosyncrasies  and  situational  expectancies  as  they  are 
Idiosyncrasies  Inherent  In  1 he  given  scene  Itself,  However,  It  Is  possible 
1o  dichotomize  the  type  scone  into  a realistic  scene  and  a stylized  scene. 
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The  realistic  scene  Is  that  which  approaches  the  natural  scene.  For  example, 
a tree  that  looks  like  a tree.  Whereas  the  stylized  scene  would  be  a scene 
in  which  an  object  is  representative  of  a given  natural  object  but  does 
not  necessarily  look  precise! v like  the  object.  For  example,  here  a pole 
with  a circle  or  sohere  on  top  would  be  Identified  as  tree.  A second 
illustration  would  be  that  In  which  a stylized  drawino  of  a triangle  is 
Identified  as  an  airplane  while  the  actual  picture  or  Generated  display 
of  an  airnlane  with  characteristics  maklna  it  look  like  an  airnlane  (l.e., 
wings,  empennane,  etc.,  ) would  be  the  more  realistic  of  the  two.  Obviously 
this  requires  judgment  on  the  part  of  pilots  or  other  aroups  of  people  to 
cateaorlze  given  olctures,  objects,  etc,  as  belno  either  more  or  less 
stylized  or  more  or  less  realistic. 

The  question  is  next  raised  as  to  what  texture  Is  to  reoresent  and 
how  texture  itself  Is  to  be  represented  In  the  scene.  Texture  Is  another 
of  these  variables  which  would  require  a molar  approach  In  the  Initial 
stanes  In  order  to  make  some  determination  as  to  later  ouantl f I cation  for 
purposes  of  definition.  At  this  point,  to  illustrate  the  general  variable, 
one  might  propose  that  texture  could  be  dichotomized  Into  those  texture 
which  are  stochastic  and  those  which  are  deterministic.  This,  of  course, 
would  allow  for  experimentation  on  the  basis  of  whether  a texture  was 
stochastic,  l.e.,  natural  or  random  or  deterministic. 

According  to  Pickett^  texture  may  be  defined  in  two  quite  different 
ways;  as  an  abstract  optical  deslon  and  as  a visible  property  of  materials. 
There  are  arivantanes  to  each  definition  and  the  abstract  definition  provides 
a basis  for  the  purely  mathematical  or  statistical  specification  of  texture 
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and  yields  a firm  basis  for  agreement  on  what  the  stimulus  Is  that  we  are 
talking  about,  according  to  Pickett.  Considering  the  variable,  texture. 

It  could  be  said  that  it  is  perhaps  one  of  the  more  Important  variables  to 
be  included  for  investigation  In  the  category  of  image  content.  This  is 
because  texture  perception  provides  basic,  physical  information  with  regard 
to  the  size,  shape,  position,  and  orientation  of  objects,  surfaces  and  other 
batches  of  visible  substance  (cf  Pickett6).  Further,  there  are  some 
indications  that  where  molar  patterns  of  color,  shading,  or  shape  are 
minimized  or  totally  absent,  texture  may  be  a very  critical  source  of 
Information  for  determining  the  shape  of  a surface  and  the  position  and 
orientation  of  objects.  Certainly  It  has  proven  useful  In  providing 
information  about  the  relative  distance  of  objects  and  parts  of  objects 
from  the  viewer.  For  these  reasons  and  for  others  which  might  be  derived 
readily,  it  Is  apparent  that  texture  is  an  Important  variable  to  be 
researched  In  any  study  of  images  of  visual  simulation,  particularly  In 
providing  rate  Information, 

The  variable  that  has  lent  Itself  to  a more  precise  ouantl f I catl on  Is 
the  variable  of  detail,  Two  approaches  have  been  taken  for  the  auant I f I cat  ion 
of  this  variable  in  visual  simulation.  The  approach  that  has  been  used  by 
General  Electric  in  their  computer  Image  generations  has  been  that  detail 
might  be  specified  In  terms  of  range,  They  have  Indicated  that  in  switching 
from  model  levels  of  detail,  range  is  the  principle  criterion  and  they  have 
established  range  IlmlTs  which  would  be  used  to  specify  the  amount  of 
detail  which  Is  required.  This  Is  based  prlmarl ly  on  the  fact  that 
as  distance  from  bulldlnns  or  other  objects  is  Increased  the  small 
features  become  quite  small  on  a display  screen  and  when  they  become 
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comparable  to  the  display  element  size  they  no  longer  provide  visual 
cues  of  any  significance.  Thus,  level  of  detail,  as  It  were,  fades  as 
the  subjective  distance  from  the  displayed  object  Is  Increased,  As  an 
illustration,  thev  propose  that  the  first  level  of  detail  might  be  for 
close  proximity  up  to  about  1,000  ft  distances  from  the  object,  a second 
level  might  be  used  from  1,000  to  3,000  ft,  a third  level  from  3,000  to 
12,000  ft.  They  suggest  that  beyond  12,000  ft  the  object  would  no  longer 
be  selected  for  processing. 

As  an  alternative  to  the  C.E,  approach  this  study  examined  the  concept 
that  detail  mav  be  guantified  in  terms  of  the  number  of  faces  crossed  by  a 

horizontal  line  in  the  central  viewing  area  of  the  display.  Additionally, 

y 

the  average  number  of  places  for  adjacent  lines  may  be  used  as  aj;*1'at I ve I y 
stable  measure  over  different  scenes.  For  a scene  oescr I nearby  the  edges 
bounding  faces,  the  number  of  stop-after-start  edges  I s -a  count  of  the 
number  of  faces  and  hence  Is  also  a measure  of  detail.  Using  the  computer 
perorated  display  as  an  example,  the  number  of  stops-a f ter-starts  per  line 
or  average  number  of  lines  will  describe  the  amount  of  detail  of  the  Image. 
It  should  bo  emphasized  that  this  says  nothlm  about  the  so-called 
meanlmfulness  or  rea  II  sm  of  the  display.  Also,  this  shou  Id  apply  enua  I I y 
to  any  method  or  system  of  display  presentation,  be  It  computer  Image 
generation  or  film  display. 

Flnuro  '2,2  Is  a block  diagram  of  a generalized  experimental  design  of 
the  type  which  could  be  used  to  test  the  variables  subsumed  under  the 
heading  of  I mane  Content,  As  can  be  seen  In  this  design  several  approachos 
may  be  taken  to  the  empirical  study  of  the  variables.  For  example. 


Fiaure  2.2  Block  dlaqrar  of  a Tyoe-Expertnental  Desian 


before  a final  Inclusion  of  a pattern  or  tyDe  of  texture  one  mlaht  perform 
sevcrni  experiments  to  determine  an  optimal  stochastic  texture  and  optimal 
deterministic  texture  which  could  then  be  compared  with  a no  texture 
situation  under  two  levels  of  realism,  !,e.,  a realistic  design  versus  a 
stylized  deslon,  The  same  general  approach  might  be  taken  for  the  variable 
realism  In  that  several  experiments  might  be  accomplished  In  the  laboratory 
to  determine  the  level  of  realism  versus  the  level  of  stylization  which 
night  be  used  in  an  overall  deslpn  In  the  same  wav  as  with  texture.  Obviously, 
It  Is  reasonable  to  assume  that  any  of  these  variables  other  than  the  fojr 
main  effects  of  film,  transparency,  model,  and  electronic  imaoe  generation 
might  be  used  in  the  same  way.  That  is,  the  variables  could  be  explored 
Independently  before  inclusion  in  this  final  overall  deslpn. 

The  deslon  also  allows  for  the  tostlno  of  the  variables  listed  In  the 
four  Inane  oeneration  approaches.  Thus,  If  electronic  Image  generation  Is 
available  as  Is  the  case  with  the  ASIJPT  pronram  then  this  slice  may  bo 
taken  from  the  oeneral  design  and  tested.  Other  designs  might  be  tested  In 
other  wavs  at  other  places  or  if  the  general  simulator  configuration  will 
allow  films,  t ransparenc I es , and  models  might  also  be  Included  for  final 
testlno.  Thus,  the  deslon  allows  for  the  testing  of  method  of  presentation 
alono  with  the  content  variables  listed  In  order  to  test  both  main  effects 
and  the  Interactions  o<  display  type,  texture,  the  amount  of  complexity, 
the  amount  of  detail  and  whether  the  Imaoe  should  be  stylized  or  realistic. 

It  should  be  emphasized  once  aoaln  that  variables  listed  at  this 
time  In  this  experimental  deslon  are  not  proposed  as  the  ultimate 
variables.  At  some  point  In  a tostlno  program  It  might  be  manifest  that 


these  variables  are  meaningless  Insofar  as  experimentation  Is  concerned 
It  and  that  one  or  more  than  one  of  these  mloht  add  very  little  to  the 

overall  consideration  of  Imaae  content.  However,  on  balance  It  is 
felt  that  these  variables  do  merit  at  least  a preliminary  Investigation 
as  to  their  importance  In  the  overall  scheme  of  Imaqe  presentation  for 
visual  simulations, 

Another  consideration  wnlch  has  not  been  presented  In  figure  2*2  Is 
that  each  of  these  blocks  could  be  tested  also  In  the  various  maneuvers 
whi ch  the  pi  lot  wi 1 1 be  required  to  accomplish  In  the  simulator,  Taylor 
et  al,  in  their  study  to  determine  regul remen +s  for  undergraduate  pliot 
training  simulation  systems  Indicated  that  there  are  various  maneuvers 
which  will  be  regulred  of  the  pilot  to  include  taxiing,  take-off  and  climb, 
approach  and  landing,  air  work  and  aerobatics,  formation  flying,  navigation 
and  low  level  flying,  and  nloht  flying.  To  adequately  determine  the 
effects  and  make  final  determinations  as  to  workable  content  variables, 
the  design  needs  to  bo  tested  under  selected  maneuvers.  It  Is  most  probably 
true  that  certain  of  these  variables  have  greater  Importance  for  maneuvers 
such  as  take-off  and  climb  and  approach  and  landlnq  and  tnat  there  would 
be  an  Interaction  with  the  maneuver  type  and  the  genera!  content  of  the 
visual  Imago,  For  example,  night  flying  would  likely  reaulre  little  or 
no  texture  but  would  require  the  Inclusion  of  point  light  souces,  while 
take-off  and  climb  perhaps  would  reoulro  a moderate  to  complex  content 
and  a moderate  amount  cf  detail  with  some  sort  of  texture  presented  either 
rea I I st lea  1 I v or  stylized.  Thus,  maneuver  becomes  an  Important  variable 
to  be  considered  when  acplvlng  the  overall  experimental  design  to  make 
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so me  determination  as  to  the  exact  content  which  should  be  Included  In 
the  visual  Image, 

2.2.5  Fstablishing  the  Trainee's  Visual  Fnvironment 

The  trainee's  visual  environment  as  used  In  this  context  Is  that 
sol'd  visual  angle  or  area  which  he  samples  as  he  performs  the  various 
training  maneuvers.  It  particularly  Includes  the  objects  and  points 
unon  which  he  fixates  In  the  performance  of  these  maneuvers.  In  light  of 
the  emphasis  upon  the  visual  display  content  variable  It  was  deemed 
hinhly  desirable  to  obtain  pilot's  eye  fixation  data  during  training 
maneuvers  for  use  in  describing  the  visual  content  variable  In  simulation 
research  and  postulating  wavs  of  quantifying  It. 

The  increased  interest  in  simulating  the  pilot's  visual  environment 
has  focused  attention  ucon  ways  of  determining  what  the  display  should 
contain,  One  source  of  data  has  been  eye  movement  records  of  pilots 
performing  representative  flight  regimes  with  eye  movement  recording  devices, 
It  appears  feasible  to  obtain  data  from  which  to  plot  the  solid  viewing 
angle  of  the  pilot  and  to  obtain  Information  about  his  sources  of 
control  Information,  Therefore,  during  the  course  of  this  project  It 
was  recommended  that  records  of  tne  eye  movements  of  the  air  Force  under- 
graduate pilot  instructors  be  obtained  for  use  In  defining  the  visual 
environment  necessary  for  the  trainee. 

In  collecting  visual  environment  data  It  was  recommended  that  given 
maneuvers  be  accomplished  by  experienced  Instructor  pilots  with  Immediate 
playback  and  review  of  the  maneuver.  There  were  two  major  reasons  for 
recommending  an  interview  and  review  of  the  record  as  soon  as  possible  after 
tilt;  maneuvers  were  performed.  First,  the  eye  movement  recorder  recommended 
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for  use  was  eauipped  with  a 60°  field  of  view  lens.  Since  the  observer 
has  the  capability  of  extendtno  his  own  visual  tleld  of  view  beyond  that 
ootnt  throuoh  peripheral  vision,  it  is  quite  probable  that  information  Is 
available  and  utilized  bv  the  pilot  which  is  not  recorded.  The  Interview 
and  review  of  the  record  should  cover  this  possibility,  i.e,,  the  extent 
that  the  nllot  can  recall  or  elaborate  upon  peripheral  cues  which  were 
not  visible  in  the  recording. 

Second,  although  the  line  of  repard  of  the  pilot  can  be  established 
with  the  eve  movement  recorder  recommended  this  would  not  establish  the 
fact  of  the  object  or  area  on  which  the  eve  was  focused.  That  Is  to  say, 
that  the  eve  restlnr,  upon  an  object  would  not  necessarily  mean  that  It 
was  focused  upon  it.  The  pilot  should  review  the  recording,  therefore, 
and  identify  what  objects  or  areas  he  was  focused  upon  durlno  his  points 
of  recorded  fixation. 

Documentation  of  eve  point  data  In  the  total  field  of  view  was  seen 
not  on  I v as  being  of  material  assistance  In  descrlblnp  and  operationally 
definino  the  content  variable  for  visual  display  research  but  could  be 
utilized  In  other  procrams,  for  example,  the  data  couid  be  used  dtrectly 
by  instructor  pilots  in  direction  the  students  attention  to  particular 
Informational  area  In  the  visual  field  durlno  tralnino.  Primarily, 
however,  the  records  in  the  Inter. ‘ews  were  Intended  to  play  an  Important 
part  in  estab I i sh I nn  tha  optimal  field  of  view  and  the  Imane  content  for 
the  maneuvers  recorded.  These  data  were  then  to  be  used  to  structure 
experiments  deal  inn  wl+h  the  recut romonts  for  visual  simulation  In  ground 
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The  role  of  the  eye  movement  record  I nos  fn  an  overall  simulator 
research  propram  Is  shown  dlaqranatically  In  f'loure  2,3,  An  MAC  Eye 
Mark  Recorder  was  obtained  bv  Life  Sciences,  Inc,  and  transferred  to  the 
Human  Resources  Laboratory,  Living  Tralnlnq  Division,  AFSC,  at  Williams 
Air  Force  Base,  Arizona,  There  HRL/FT  personnel  developed  a means  for 
obtalnlno  and  recordlno  pilots'  eye  movements  durlno  flight.  However, 
development  and  use  of  the  technique  did  not  nronress  beyond  this  stage 
and  no  data  were  obtained  which  could  be  used  in  the  identification  of 
visual  display  content. 

2,3  MOTION-VISION  INTERACTION 

The  problem  of  the  interaction  between  information  provided  by  the 
visual  external  world  display  and  the  motion  platform  is  a dual  one. 

In  the  first  case,  lack  o*  correspondence  between  the  two  sources  of 
Information  cause  severe  protlems  to  the  pilot  in  his  control  of  the 
svstem  and  provide  a poor  learning  situation.  This  is  particularly  true 
when  the  synchronization  between  the  Information  from  the  two  sources  Is 
lacking.  Indications  from  other  systems  are  that  as  little  as  100  milli- 
seconds of  time  Ian  between  the  motion  Information  and  That  provided  by 
the  visual  system  presents  severe  difficulties  for  control  of  the  system. 

In  the  second  case,  the  required  degree  of  fidelity  of  motion  may  not  be 
as  stringent  if  a nood  display  svstem  Is  present.  It  Is  a testable 
hypothesis  that  sllnht  rates  of  onset  of  motion  when  combined  with  good 
visual  display  provides  perceptually  realistic  Information  for  control 
of  the  svsten.  That  is,  the  requirements  for  a high  fidelity  motion  system 
is  lessened  with  a oood  visual  system. 
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Finure  2.5  Program  of  events  !n  testing  performance  equivalence  and  use  of  the  resui+s 


Flqure  2,3  Continued 


2,3,1  The  Ef fee five  Time  Constant  Concept 

The  motion-vision  interaction  problem  has  been  examined  in  the  light 
of  the  effective  time  constant  (te)  concept  developed  by  Matheny  and 
Norman^  in  connection  with  the  description  of  psychomotor  behavior.  Under 
this  concept  a testable  hypothesis  Is  that  the  efficiency  of  control 
behavior  is  a function  of  the  time  laa  between  the  initiation  of  the  control 
response  and  the  Immediacy  of  information  feedback  of  the  results  of  that 
response  to  the  controller.  The  feedback  may  cone  thronah  any  of  the  human 
senses  but  in  control  of  moving  systems  the  primary  ones  are  those  of 
motion  and  vision,  Cor  systems  with  freauancy  responses  in  the  range  of 
aircraft,  the  notion  senses  perceive  rates  of  onset  of  accelerations  and 
acce I erat Ions  of  the  system  in  advance  of  those  changes  In  the  svstem  which 
are  capable  of  being  nercelved  by  the  visual  sense.  Thus,  when  the  pilot 
Is  flying  on  instruments  in  which  the  index  which  he  Is  controlling  may 
move  very  little  per  unit  displacement  of  the  aircraft  compared  to  the 
outside  visual  display,  the  motion  cues  are  sensed  well  ahead  of  their 
visual  counterparts.  When  one  moves  from  Instrument  to  contact  flight 
the  gain  of  the  visual  display  Is  tremendously  Increased,  For  example, 
changes  in  aircraft  pitch  may  be  sensed  In  the  contact  scene  much  more 
ouicklv  than  through  watching  the  ar+ifical  horizon.  The  increased  gain 
In  the  visual  display  makes  apparent  any  discrepancies  between  the  motion 
and  the  visual  Information  and.  In  particular,  any  lack  of  synchronization 
or  phase  differences  between  the  two. 

The  effective  time  constant  of  a man-machine  system  Is  a function  of 
the  threshold  value  of  the  sensory  mode  of  the  operator.  Rate  of  onset 
and  acceleration  Information  are  perceived  by  the  motion  sensors  well  In 
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advance  of  the  rate  and  position  Information  sensed  by  the  visual  modality, 
Howevor,  changes  In  gain  In  visual  display  can  lower  the  visual  thresholds 
for  detecting  these  movements  which  Is  In  the  case  in  high  resolution  large 
contact  displays.  However,  those  characteristics  of  the  contact  visual 
display  which  tend  to  occlude  Its  resolution  and  raise  the  threshold  of 
detection  on  the  oarf  of  the  operator  will  Increase  the  effective  time 
constant  and  can  be  hypothesized  to  increase  his  dependence  upon  motion 
cues. 
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3.0  MEASUREMENT  AND  METHODOLOGICAL  CONS  I DERATIONS 

Since  one  of  the  responsibilities  under  the  overall  program  was  the 
design  and  detailed  description  of  experiments  relevant  to  the  experimental 
ouestions,  much  consideration  was  oiven  to  certain  methodological  problems 
underlvlnn  this  type  of  research.  These  problems  fall  under  the  general 
headings  of  (I)  classical  transfer  of  tratnlng  paradigm  versus  performance 
equivalence  experimentation,  (2)  performance  measurement,  and  (3) 
economical  experimental  designs. 

Transfer  of  training  research  has  the  ooal  of  producing  Information 
which  will  be  useful  in  dec  .'ns  reaardtno  the  cost  effectiveness  of 
trainina  methods,  procedures  and  devices.  What  is  souoht  Is  maximizing 
(I)  the  efficiency  of  the  training  system  and  (2)  the  transfer  of  train- 
ing between  that  system  and  the  criterion  system.  Maximum  effectiveness 
of  +ha  training  system  Is  sought  with  a minimum  complexity  of  training 
procedures  and  egulpment  (particularly  the  simulator).  Once  the  efficiency 
and  effectiveness  of  the  training  system  has  been  determined  Its  cost 
effectiveness  can  be  calculated  through  Incorporation  of  current  cost 
Information, 

It  has  been  traditional  to  evaluate  a tralnlnq  system  solely  on  the 
basis  of  the  degree  to  which  training  In  that  system  transfers  to  the 
criterion  svstem.  The  Importance  of  separating  for  study  the  ouestions 
of  the  effectiveness  of  the  training  system  and  the  amount  of  transfer 
from  that  system  to  the  criterion  system  has  traditionally  not  been 
emphasized.  The  approach  taken  In  this  present  work  was  to  take  advantage 
of  the  capabilities  of  the  ASUPT  to  develop  It  Into  a criterion  system 
epuivalent  to  the  T-37  training  aircraft  In  terms  of  pilot  performance. 
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Research  aimed  at  determining  the  most  effective  methods  and  media  for 
training  would  then  use  ASUPT  as  the  criterion  system.  That  Is  to  say 
that  the  full  capability  ASUPT  In  which  performance  Is  equivalent  to 
the  T-37  would  be  used  to  determine  those  minimum  characteristics  of  the 
simulator  sufficient  for  training  In  particular  units  or  phases  of 
Instruction,  procedures,  instructor-student  Interaction  and  automated 
training  techniques  can  be  studied  In  the  ASUPT  to  arrive  at  the  most 
efficient  tralnlnq  system.  The  training  systems  thus  derived  for 
particular  units  or  phases  Instruction  could  then  be  used  In  a small 
number  of  transfer  of  tralnlnq  experiments  to  aetermlne  their  effectiveness 
In  training  for  the  final  criterion  system  - the  T-37  aircraft. 

The  concept  of  efficient  and  effective  training  systems  Is  Illus- 
trated when  two  different  training  systems  result  In  the  same  transfer  of 
training  to  the  criterion  system  but  accomplish  the  training  through 
different  methods,  procedures  or  training  equipment.  One  training 
system  may  be  much  more  efficient  than  the  other  In  terms  of  Its  use  of 
trainee  and  Instructor  time,  complex  equipment  and  procedures.  Conversely, 
two  equally  efficient  training  systems  may  exhibit  differential  transfer 
to  the  criterion  system.  That  is  to  say,  that  a student  might  progress  to 
criterion  performance  In  one  system  as  quickly  and  efficiently  as  In 
another,  however;  the  two  systems  may  not  be  equivalent  In  terms  of  the 
transfer  of  the  training  from  them  Into  the  criterion  system. 

The  distinction  between  the  efficiency  of  the  training  system  and  the 
degree  to  which  training  In  that  system  transfers  to  the  criterion  system 
Is  a concept  which  Is  fundamental  to  the  experimental  approach  and 
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methodology  being  develoned  In  this  work.  It  Is  contended  that  the 
>i  classical  -transfer  of  tralnlno  experiment,  aside  from  Its  other  In- 

conveniences, Is  not  an  effective  method  for  determining  the  efficiency 
of  the  traininq  system.  This  can  best  be  done  after  establishing 
performance  equivalence  between  the  ASUPT  and  the  T-37  aircraft  and  then 
using  AStJPT  as  the  criterion  system.  The  more  classical  transfer  of 
tralnlno  experiments  would  then  be  limited  to  the  final  question  of  the 
degree  of  transfer  to  be  obtained  from  maximally  efficient  system  derived 
from  research  using  the  performance  equivalence  approach,  Further,  and 
just  as  Importantly,  the  performance  equivalence  procedure  may  be  used  to 
eva 1 ua+e  the  adequacy  with  which  any  given  simulator  represents  the  air- 
craft in  terns  of  required  control  behavior, 

3.1  CLASSICAL  TRANSFER  OF  TRAINING  PAPAD I G‘ • 

The  classical  transfer  of  tralnlno  experiment  Is  designed  to  deter- 
mine the  differences  In  traininq  time  on  the  criterion  task  as  a result 
of  traininq  In  two  different  situations.  Typically,  time  to  attain 
criterion  in  the  criterion  task  after  tralnlno  under  an  experimental 
condition  is  compared  to  conventional  vra Inina.  It  Is  Important  to  note 
that  in  sett  Inn  up  the  new  or  experimental  training  system  numerous 
alternatives  with  respect  to  simulator  conf I nurat Ion,  training  methods, 
procedures  and  curricula  will  have  been  considered  and  chosen  among. 

The  classical  transfer  of  tralnlno  paradigm  offers  only  a very  expensive) 
means  of  detormlnlnq  which  combination  of  those  Is  the  most  efficient  for 
attalnino  that  level  of  performance  which  may  be  set  as  a prerequisite 
| for  transferrlnn  to  the  operational  system, 
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It  Is  Important  to  examine  the  problem  of  deciding  what  the  experi- 
mental conf Iguratlon  of  the  training  simulator  should  be  In  the  classical 
transfer  of  training  paradigm,  Before  such  an  experiment  can  be  performed 
it  is  necessary  to  define  the  range  and  levels  of  the  simulator  variables 
to  be  used  in  the  experiment  In  a manner  congruent  with  the  perceptual 
capabilities  of  the  human  operator.  It  Is  necessary  to  determine  over 
what  range  anv  physically  defined  parame+er  of  the  simulator  is  perceptable 
to  the  operator  and,  within  this  range  what  the  dlscrlmtnable  Intervals  of 
that  parameter  are. 

By  way  of  example,  It  Is  possible  to  define  a range  of  simulator 
platform  motion  and  to  designate  various  levels  of  categories  within  that 
ranae  which  may  be  considered  as  levels  of  the  experimental  variable.  At 
the  same  time,  however,  those  Interval;  which  have  been  decided  upon  on  the 
basis  of  some  physical ly  measurable  parameter  of  motion  may  be  Ind I st Inau I s- 
able  to  the  human  operator.  Thus,  each  classical  transfer  of  training 
experiment  deal  Inn  with  characteristics  of  the  training  simulator  should 
logically  be  preceded  by  experiments  establishing  the  perceptual  range 
and  dlscrlmlnable  Intervals  of  the  physical  variable  under  study. 

The  suggested  alternative  to  the  classical  transfer  of  training 
paradigm  Is  that  which  we  have  termed  the  performance  eoulvalence  approach. 
This  approach  concept  Is  discussed  In  the  following  section, 

3.2  THE  PCRFOPNANCE  EQUIVALENCE  RESEARCH  APPROACH 

As  stated  In  the  previous  section  the  classical  transfer  of  training 
experiment  Is  time  consuming,  expensive  and  subject  to  a number  of 
constraints.  If  one  apparently  reasonable  assumption  Is  made,  the  need 
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for  the  classical  transfer  experiment  may  be  eliminated.  This  assumption 
Is  that  the  transfer  of  tralnlnn  between  the  simulator  and  the  aircraft 
will  be  high  and  positive  If  the  simulator  Is  eaulvalent  to  the  aircraft 
In  terns  of  performance  reaulrements  on  the  operator.  This  means  that 
the  measured  nan-machine  system  output  and  the  operator  performance  or 
Input  into  the  simulator  controls  are  not  different  from  those  exhibited 
In  the  aircraft  during  the  performance  of  plven  maneuvers  under  given 
conditions.  How  performance  enulvalence  Is  established  and  what  It  means 
both  In  terms  of  determining  the  reaulrements  for  the  tralnlnn  simulator 
and  its  usefulness  in  research  deal  Inn  with  training  methods  Is  the  subject 
of  this  section. 

The  performance  enulvalence  approach  to  the  transfer  of  training 
problem  requires  that  the  performance  of  both  the  man-machine  system  and 
the  operator  he  determined  In  the  criterion  vehicle.  In  the  case  of 
motion-vision  research  with  the  ASUPT  this  criterion  vehicle  Is  a T-37 
tralnlnq  aircraft.  The  performance  measures  for  the  man-machine  system 
and  the  operator  which  are  required  In  order  to  establish  performance 
enulvalence  are  discussed  In  Section  3,3  below. 

Once  performance  equivalence  has  been  determined  over  the  range  of 
aircraft  maneuvers  of  Interest  and  under  reoresentat Ive  conditions  of 
other  variables,  e.q.,  turbulence,  the  performance  equivalent  simulator 
may  bo  used  for  two  distinct  purposes.  These  ore  (I)  the  determination 
of  the  minimum  set  of  characteristics  of  the  simulator  which  are 
equivalent  In  terms  of  performance  to  the  aircraft  and  (2)  Its  use  as  a 
criterion  system  In  research  on  the  most  efficient  methods  of  training, 


simulator  utilization.  Instructor-student  Interaction,  student  motivation, 
and  a host  of  other  Important  tralnlnp  nuestlons. 

It  Is  not  suggested  that  the  performance  eoul valence  approach  to  the 
transfer  of  tralnino  question  will  result  In  that  configuration  of  the 
simulator  which  will  maximize  transter  of  tralnlnp  since  It  Is  quite 
possible  that  simulator  characteristics  which  are  much  more  "difficult" 
to  control  mipht  result  !n  a greater  amount  of  transfer  to  the  criterion 
aircraft.  At  the  same  tine,  however,  such  a simulator  might  not  De  the 
conf ! nurat I on  which  Is  best  for  most  efficient  training  In  the  training 
system. 

The  performance  equivalence  approach  Is  recommended  for  establishing 
the  basic  or  minimum  character i st I cs  for  the  tralnlnq  simulator  and,  using 
this  equivalent  svsten,  the  establlshlno  of  the  most  effective  combinations 
of  methods,  procedures  and  other  training  media  for  bringing  the  trainee 
to  criterion  performance  in  the  training  system.  This  training  system 
could  then  be  used  In  a definitive  transfer  of  training  experiment  to 
determine  the  savInas  In  aircraft  training  time  brought  about  through 
uslno  the  training  system. 

3.3  PU Rr OR* 'AN CF  "EACUREb’ENT 

One  of  the  first  guest  Ions  to  be  answered  In  the  comparison  of  two 
systems  using  performance  equivalence  Is  the  manner  In  which  the 
Performance  Is  measured.  Traditional  measures  for  system  output  exist 
and  a good  deal  Is  known  about  their  Inter-relationships,  In  the  air- 
craft svstem  such  measures  as  altitude,  airspeed,  rate  of  climb  and  bank 
and  oltch  angle  are  measures  which  reflect  the  performance  of  the  man- 
machine  combination,  (Those  system  output  measures  are  shown  as  PP^  In 


Ftnurc  3,1)  As  the  human  operator  moves  from  one  system  to  the  other  he 
can  adant,  within  limits,  to  the  control  renuirements  of  the  svstem  In 


Figure  3.1  Performance  Measurement  Points  (MP)  In  the  Man/Machine  System 


order  to  make  It  perform  to  a certain  level  of  svster1  output,  (The  operator's 
control  input  behavior  Is  shown  as  MPj  In  Flqurp  3,1.)  For  example,  the 
trained  pilot  can  adapt  to  the  nulte  dissimilar  control  renuirements  of  a 
helicopter  and  a fixed  wing  aircraft  In  producing  the  same  svstem  output 
for  the  two  vehicles.  That  Is  to  say  that  he  can  adapt  his  control  behavior 
nuite  markealv  In  order  to  make  the  vehicles  perform  In  Identical  fashion. 
Similarity  the  operator  of  an  automobile  can  adapt  his  control  Input  behavior 
to  successfully  cope  with  the  differences  In  steerlnn  ratio  and  response 
dynamics  of  the  larqe  passenner  vehicle  and  the  small  sports  car.  It 
should  bo  no+od  here  that  this  adaptive  behavior  Is  critical  to  transfer 
of  tralnlnn.  The  transfer  of  tralnlno  literature  and  experience  Indicate 
that  positive  transfer  of  tralnlnn  can  and  does  take  place  from  a vast 
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variety  of  sources  and  modes  of  training.  These  range  from  cognitive 
rehearsal  to  practice  In  the  actual  aircraft.  The  central  point  Is  that 
the  trainee  can  adapt  his  renertoire  of  behaviors  to  the  retirements  of 
the  svster  but  adaptation  takes  tine.  The  more  near  I v his  learned  responses 
are  like  those  retired  In  the  criterion  system  the  less  the  adaptation 
time  will  be  expected  to  be.  The  training  literature  also  Indicates  that 
the  trainee  would  exnerlence  creates  difficulty  In  adantino  to  a 
chanced  response  than  to  a chanced  stimulus.  Thus,  the  Importance  of 
attention  to  the  control  Input  behavior  being  shaped  by  a nlven  tralnlnc 
device  can  not  be  overemphasi zed, 

"easuros  which  reflect  how  the  operator  adapts  his  control  behavior  to 
chances  in  the  control  element  dyncHcs  have  been  the  subject  of  I nvest  I cat  Ion 
'or  some  time  althounh  not  as  Intensively  as  have  system  output  measures. 

One  of  the  earliest  serious  efforts  In  this  regard  was  the  work  of  F|tts, 

8 

Bennett  and  Bahrick  , Fitts  et  al.  were  Interested  In  descriptions  of  the 

trackinn  control  behavior  of  the  operator  In  closed-loop  dynamic  systems, 

f'uch  of  the  work  concerned  with  operator  Input  behavior  has  followed  the 

example  of  Fitts  and  has  looked  at  ways  of  describing  the  frecuency  and 

extent  of  the  operator's  control  Inputs,  A summary  of  these  Investigations 
, , . 9 

Is  contained  In  the  report  by  Matheny  prepared  and  presented  during  the 
course  of  this  project., 

during  the  course  of  this  project  a great  deal  of  effort  was 
concentrated  upon  an  attempt  to  gather  data  to  develop  operator  output 
measures  and  test  the  feasibility  of  the  performance  enu I valence  concept. 

An  unusual  potential  for  doinn  this  was  afforded  by  the  existence  of  an 
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i nstrunonted  T-37  training  aircraft  and  the  ASUPT  simulator  system  for 
that  al rcraft. 

As  indicated  in  the  Introduction  to  this  report  an  overall  program 
of  pilot  tralnlnn  research  in  motion  and  visual  display  simulators  has 
a major  aim  of  this  project.  The  major  features  and  Interrelationship 
of  the  pronram  developed  during  this  project  is  shown  In  Flaure  3,2. 

Wh,le  this  pronram  is  discussed  in  more  detail  in  Section  4,0  It  is 
presented  here  to  Indicate  the  pivotal  role  desianed  to  be  played  by  the 
data  collected  from  the  T-37  aircraft. 

As  shown  in  Figure  3,2  one  of  the  first  steps  In  comparing  the  ASUPT 
and  T-37  aircraft  systems  was  to  collect  data  In  both  the  aircraft  and 
the  ASUPT,  From  these  data  the  quantitative  description  of  the  pilots 
control  behavior  in  fiylna  the  aircraft  was  projected  to  be  developed  for 
use  In  makino  a performance  equivalence  comparison  to  the  ASUPT. 

The  collecting  of  data  in  the  T-37  aircraft  was  troubled  by  both 
administrative  and  instrumentation  problems.  Administrative  problems 
centered  primarily  unon  where  the  flights  would  be  flown,  l,e,,  Wrlaht- 
Patterson  Air  Force  Rase  or  Williams  Air  Force  Base.  The  obvious  Instru- 
mentation problems  were  surmounted  without  great  difficulty,  Others  could 
not  be  detected  and  isolated  until  the  detailed  analysis  of  the  data  was 
undertaken.  These  problems  are  discussed  In  Appendix  A which  deals  with 
the  analyses  conducted  within  the  limited  time  frame  left  for  such  analysis 
after  T-37  aircraft  data  collection  was  accomplished. 

Data  from  the  ASUPT  for  use  in  comparing  the  ASUPT  and  T-37  system  was 
limited  to  one  flight  due  to  a number  of  technical  problems,  priorities  and 
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coordination  with  on.qolng  activities  at  Human  Resources  Laboratory, 

Flylnq  Training  Division. 

In  summary  of  the  oroqress  in  developing  measures  of  pilot  control 
behavior  and  test  of  the  performance  equivalence  concept,  while  date 
was  collected  In  the  T-37  aircraft,  certain  instrumentation  anomalies 
were  present  in  the  data  which  require  more  extensive  data  handling  before 
analysis  than  was  expected.  The  potential  usefulness  of  the  data  Is  as 
qreat  as  original iy  anticipated  for  developlna  descriptions  of  pilot 
behavior.  With  respect  to  comparisons  with  ASUPT  these  are  placed  in 
doubt  by  the  unexpected  report  by  key  personnel  working  with  ASUPT  that 
it  could  not  be  modified  to  accommodate  the  tests  necessary.  Although 
this  conclusion  needs  further  examination  the  necessary  time  was  not 
available  to  make  pronress  in  this  area. 

The  concent  of  performance  equivalence  can  not  be  said  to  have  been 
tested  from  the  data  analysis  conducted  to  the  time  of  wrltlna  this 
report.  Delays  In  collectlnn  the  aircraft  data  allowed  tlm©  for  only  the 
beglnnlnn  analysis  described  In  Appendix  A,  These  analyses  Indicate  that 
more  complete  analysis  would  he  most  fruitful  for  (I)  Identlfylm  the 
descriptive  coefficients  for  the  pilots  behavior  useful  for  mokinc 
comparisons  between  aircraft  and  simulator,  (2)  delineating  the  conditions 
under  which  ASUPT  measu-es  should  be  taken  for  studying  comparability  of 
the  two  svstems,  (3)  providing  normative  data  as  to  expected  student 
criterion  performance  for  both  system  output  and  operator  input  measures 
and  (4)  provide  operator  Input  measuremept  techniques  for  use  In  developing 
teaching  techniques,  either  conventional  or  adaptive,  which  concentrates 
upon  shaping  the  trainees  control  Input  behavior. 


4.0  MOTION-VISION  RESEARCH  PROGRAM 

The  I nvest I cat  Ion  of  vision,  motion  and  their  Interactive  effects  was 
considered  as  an  overall  program  of  research  In  which  the  ASUPT  system 
would  nlav  the  major  role.  In  brief,  the  ASUPT  system  was  nroposed  to  be 
made  equivalent  to  the  T-37  aircraft  In  terms  of  pilot  behavior  required 
to  perform  certain  maneuvers  to  specified  criteria.  The  maximum  capability 
of  the  ASUPT  would  be  utilized  In  adjusting  Its  parameters  to  confloure 
it  so  that  the  pilot  control  requirements  were  the  same  as  those  acaulred 
In  the  aircraft.  The  overall  pronram  Is  shown  d I agramat I ca I I y In  Figure 
2.3,  Page  31. 

The  sequence  of  studies  was  designed  to  Investigate  first  the  motion 
characteristics  of  the  simulator  with  only  Internal  Instruments  as  visual 
displays.  This  was  done  because  it  was  felt  that  the  application  of  the 
performance  oauivalence  concent  should  be  first  applied  to  the  situation 
In  which  the  experimental  variables,  i.e«,  motion  of  the  simulator,  could 
be  specifically  defined  with  the  visual  display  of  Information  Identical 
to  that  of  the  aircraft.  This  approach  was  also  coincident  with  the  schedule 
of  dellverv  of  ASUPT  equipment  and  Its  availability  for  research. 

As  pointed  out  In  Section  2,0  the  characteristics  of  the  motion 
Imparted  to  the  system  by  the  external  forcing  function  of  turbulence 
Is  of  extreme  Importance  In  the  study  of  motion  cues.  As  shown  In 
Flnure  2.3  the  T-37  aircraft  data  was  desianed  to  be  used  in  the  deter- 
mination of  these  forcina  functions  as  Indicated  In  the  block  entitled 
"Pilot  Motion  Environment  In  the  T-37."  These  data  were  then  to  be  used 
In  the  definition  of  motion  parameters  and  their  Implementation  In  the 
ASUPT  svstem  as  discussed  earlier.  The  block  entitled  "Describe 
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Aircraft-Pilot  System  in  Figure  2,3  and  that  entitled  "Compare  Aircraft 
^ and  A5UPT"  indicates  the  steps  and  Investigations  discussed  under  Methodology 

and  Measurement  Considerations  In  Section  3,0.  Based  upon  these  data  the 
adjustment  of  ASUPT  to  be  equivalent  to  the  T-37  aircraft  was  projected  to 
be  accomplished  followed  by  simulator  motion  requirement  studies.  The 
oeneral  design  of  these  studies  was  aimed  at  determining  the  minimum  number 
of  dimensions  and  the  minimum  deoree  of  fidelity  of  physical  simulation 
defined  in  terms  of  bandwidth  of  response  of  the  motion  platform  at  which 
performance  was  still  equivalent  to  the  full  simulation. 

In  Fioure  2.3  Is  shown  also  the  use  of  eye  movement  recordings  In 
the  annroach  to  the  definition  of  the  visual  parameters  to  be  studied  as 
visual  displav  characteristic  of  the  simulator.  The  eye  movement  data 
was  programed  to  nlav  a prominent  role  in  Identification  of  the  positions 
of  objects  in  the  external  scene  and  the  Internal  referents  used  by  the 
pilot  in  control  of  his  aircraft.  The  procedure  to  be  followed  was 
identical  to  that  in  the  study  of  motion  requirements  in  that  the  simulator 
would  be  confinured  to  be  equivalent  to  the  T-37  aircraft  In  terns  of 
pilot  performance  using  the  maximum  capability  of  the  ASUPT  device.  Assuming 
this  could  be  accomplished,  the  subsequent  experiments  would  be  aimed  at 
determining  the  minimum  necessary  and  sufficient  character  I st I cs  of  the 
visual  displav  for  bringing  about  performance  of  the  pilot  equivalent  to 
that  In  the  T-37  aircraft. 

The  extent  to  which  the  research  program  was  accomplished  has  been 
outlined  In  the  previous  Sections.  In  brief,  although  T-37  aircraft  data 
was  collected  the  delay  in  its  collection  allowed  time  only  for  the 
briefest  preliminary  analysis  of  the  data.  With  respect  to  definition 
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of  the  visual  parameters,  an  eye  movement  recorder  was  obtained  and  con- 
f Inured  for  use  In  the  then  T-37  aircraft,  Data  collection  feasibility 
was  then  tested.  However,  only  tentative  data  for  specifying  positions 
and  characteristics  of  Internal  and  external  references  were  obtained. 


5,0  CONCLUSIONS  ANO  RECOMMENDATION 


The  overall  object  of  the  project  of  Investigating  the  role 
In  the  tralnlrn  simulator  of  motion,  visual  display  and  their  Inter- 
action was  not  accomn ! I shad.  The  Important  first  steps  necessary  to 
their  Inves+loation,  l.e.,  explicit  description  and  quantification  as 
experimental  variables,  was  made  and  Is  presented. 

Due  to  the  extremely  larne  number  of  states  of  the  experimental 
variables  In  dealing  just  with  motion  and  visual  display  attention  was 
riven  In  the  greater  portion  of  the  work  to  development  and  test  of 
mo^e  economical  methods  of  experimentation.  The  concept  of  the 
performance  ecu  I valence  between  the  simulator  and  the  aircraft  was 
presented  along  with  the  arguments  for  the  importance  of  pilot  Input 
measures  In  addition  to  system  output  measures  Iri  research  on  training 
devices.  An  attempt  was  made  to  take  advantage  of  the  opportunity  fcr 
obtaining  pilot  performance  data  both  In  an  1 nst'-umented  T-37  training 
aircraft  and  the  research  simulator  of  that  aircraft,  the  ASL'PT.  Delavs 
and  difficulties  in  collecting  data  both  In  the  a) rcraft  and  In  ASUPT 
allowed  time  for  only  preliminary  but  promlslno  analysis  of  +he  aircraft 
data.  Certain  data  recording  anomalies  in  the  form  of  sourlous  signals 
make  It  necessary  to  carry  out  more  data  clean  up  than  was  anticipated. 
Nevertheless,  the  opportunity  for  developlno  and  testing  a new  approach 
to  establlshlnn  nuant I *at I ve  indices  of  the  denree  of  correspondence 
between  aircraft  and  simulator  based  upon  operator  control  behavior  Is 
present  with  these  data.  1+  Is  recommended  that  tne  opportunity  not 
be  lost  and  that  the  analyses  be  carried  on. 


In  the  conduct  of  thlc  project  an  overall  program  for  systematically 
Investigating  the  motion  and  visual  display  variables  was  outlined.  The 
carrying  out  of  the  plan  If  successful  would  provide  nu'cker  and  more 
economical  answers  to  both  simulator  characteristics  and  training 
methodology  ouostions  provided  the  necessary  funds  and  personnel  support 
could  be  brouoht  to  hear  on  the  problem.  It  Is  recommended  that  the 
pronram  be  pursued. 
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APPENDIX  A 


Summary  Progress  In  T-37  Aircraft  Pilot  Contral  Behavior  Analysis 

1.0  Purpose  of  this  appendix  Is  to  present: 

1.1  The  detailed  method  of  analysis  used  to  develop  a linear, 
time  variable  pilot  identification  for  the  T-37  aircraft 

i n-f I I oht  data . 

1.2  A critioue  of  the  method  used  and  of  the  data  available  for 
ana  lysis, 

1.3  Suggestions  for  improving  the  methodology  and  data 
preparation. 

2.0  Summarv 

2.1  The  full  sepuencn  of  the  operations  needed  to  accomplish  the 
pilot  discrete  time  state  enuatlon  Identification  as  out- 
lined in  the  addendum  to  this  appendix  has  been  Implemented, 
Five  pilot  states  were  assumed, 

2.2  A slnole  spurious  data  pulse  In  all  channels  resulted  In  a 
discernable  difference  In  trend  of  the  Identified  co- 
efficients. 

2.3  The  Identified  coefficients  show  cyclic  trends,  as  they  did 
in  preliminary  Identification  analyses, 

3.0  Conclusions 

3.1  Use  of  tha  actual  Input  numerical  data  In  the  Identification 
yields  misleading  results.  Error  values  of  the  Inputs  from 
a standard  are  necessary. 

3.2  Usurious  instrumentation  slnnals  attenuated  tho  results  of 
the  Fourier  analysts  to  some  extent  and  Indicated  the 
necessity  for  a data  smoothlnn  process. 


Si 


4.0  Recommendations  for  further  and  complete  analysis  of  the  data 


5.0 


4.1  Use  Input  error  values  In  the  Identification  rather  than 
absolute  values, 

4.2  Remove  the  spurious  slpnals  from  the  data. 

4.3  Transform  heading  Into  a continuous  seouence. 

4.4  Seek  a better  criterion  for  Fourier  Series  truncation 
to  select  fewer  terms  for  coefficient  Identification 
ana  lysis. 

4.5  If  Implementation  of  Items  (I)  through  (4)  still  yields 
raoldlv  varvlnn  coefficients,  increase  pilot  state 
dimensions,  (e.n.,  elevator  deflection  rate)  and  add 
possible  cue  signals  (e.q,,  rate  of  climb  and/or  rate  of 
onset  of  acceleration  (jerk)  until  slowly  varvlnn  Identified 
coefficients  result. 

Data  preparation  and  Identification  method, 

b.l  The  methods  used  were  those  outlined  In  Addendum  I,, 
except  as  noted  below, 

5.2  It  was  evident  from  review  of  the  summary  listings  of 
the  missions  flown  in  the  T 37— 1 948  aircraft  provided  by 
Wr ! nht-Patterson  AFB  that  there  existed  many  spurious 
roints  In  the  data.  The  summary  llstlnos  provided  a print 
out  every  2 seconds  or  at  every  40th  data  point.  These 
spurious  values  were  due  to  an  unidentified  source  which 
produced  saturated  or  near-saturated  signals  at  random  - 
appearlno  Intervals.  In  order  to  avoid  data  contamination 
a mission  was  selected  which  appeared  from  scrutiny  of  the 
summary  listings  to  contain  no  spurious  points. 


5.3  One  maneuver,  a 30°  left  bank  at  100  knots  airspeed  flown  under 
contact  conditions,  was  selected  for  prototype  analysis.  The 
summary  llstlno  for  the  maneuver  appeared  to  contain  no  anomalies 
for  any  of  the  recorded  variables, 

5.4  Twenty-two  (22)  recorded  variables  were  selected  for  analysis. 
Five  (5)  of  these  were  hypothesized  to  be  measurements  of 

pilot  state  variables  <f).  These  were  elevator,  aileron,  rudder, 
aileron  trim  tab  and  throttle  positions.  The  remaining  seventeen 
(17)  variables  were  hypothesized  to  be  measurements  of  the  pilot 
Input  variables  (u):  These  were  elevator  stick  force,  aileron, 

stick  force,  rudder  force,  airspeed,  altitude,  heading,  pitch 
angle,  pitch  rate,  roll  rate,  yaw  rate,  pitching  acceleration, 
rolling  acce I eratf on , yawing  acceleration,  and  longitudinal, 
lateral,  and  norma!  acceleration  at  the  pilot's  station.  These 
variables  were  to  be  used  to  Identify  the  pilot  by  the  discrete 


time  form: 


, — A , f + B,  v 

-|  + | I -|  _1  * 


where  Aj  and  Bj  are  slowly  time 
varying  functions  relative  to  the  length  of  the  maneuver, 

5,5  The  measurements  were  presumed  to  be  contaminated  with  noise  due 
to  Inaccurarcl es  of  recording  Instruments,  resolution  of  the 
reccrdlno  Instruments,  Inaccuracies  and  resolution  of  pilot 
control  movements,  atmospheric  turbulence  and  other  unidentified 
sources.  In  order  to  reduce  the  effect  of  random  disturbances 
upon  the  pilot  Ident I f teat  Ion  procedure,  the  raw  data  was 

processed  to  remove  some  Identifiable  randomness  and  as  a conseauence 
maximize  the  deterministic  content  of  the  data.  The  processing 
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method  was  not  Intended  to  remove  the  effects  of  the  spurious 
signals  mentioned  In  5,2,  which  appeared  to  a randomly  occur  I no 
signal  of  fixed  magnitude. 

5.6  The  power  soectral  density  of  the  first  2048  Dolnts  (out  of  a 
total  of  2400  points)  of  the  two  minute  maneuver  was  computed 
for  each  of  the  22  variables.  A fast  Fourier  transform  routine 
was  used  to  compute  the  power  spectrum.  A print  out  and  a plot 
of  the  power  spectra  of  each  of  the  22  variables  are  available 
for  Inspection  as  data  1 1 st l no  In  "Mathenv  FFT  1-22."  The 
spectra  were  hypothesized  to  be  composed  of  the  composite 
spectra  due  to  deterministic  signal  and  of  random  noise.  An 
earlier  preliminary  power  spectral  analysis  of  two  variables 
(elevator  and  altitude)  In  a stall  maneuver  from  another  mission 
It  was  determined  that  beyond  a certain  freeuency  all  the 
spectral  values  were  of  essentially  constant  maqnltude  (l.e., 
the  spectrum  was  flat).  For  those  freauencles  for  which  the 
spectrum  was  flat,  It  was  assumed  that  the  major  contribution  to 
f he  time  history  of  the  signal  was  due  to  a random  sequence. 
Since  bevond  a certain  frequency  there  Is  essentially  no 
determinism  In  the  signal,  l.e,,  It  Is  a random  sequence,  the 
signal  was  programed  to  be  smoothed  to  contain  no  frequencies 
greater  than  that  at  which  there  was  only  noise.  This  program 
Is  accomplished  by  truncatlnq  the  Fourier  series  describing  the 
variable.  This  does  not  preclude  the  existence  of  randomness 

In  the  remaining  terms  of  the  series, 

5.7  From  the  examination  of  the  power  spectra  of  the  elevator  and 
altitude  variables  described  In  (Section  5,6),  It  appeared  that 
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the  power  spectra  consisted  of  a few  iow  frequencies  of  large 
magnitude  decreasing  monoton  I ca I I y to  a flat  spectrum  of 
smaller  maonltudes.  The  average  value  of  the  power  spectrum 
over  all  freouencles  appeared  to  be  slightly  greater  than  the 
mean  value  of  the  flat  portion  of  the  spectrum.  Conseouentlv 
a criterion  was  established  for  truncation  of  the  spectrum. 

This  Involved  selection  of  the  highest  freouency  which  had  a 
spectral  value  greater  than  the  mean  spectral  value  and 
elimination  of  all  frequency  components  greater  than  that 
frequency. 

When  the  power  at  zero  freauency  was  Included  In  the 
computation  of  the  mean  the  mean  spectral  power  was  so  larae  for 
some  variables  (e.o.,  altitude)  that  only  the  value  at  zero 
frequency  had  a power  greater  than  the  mean,  Conseouently  the 
power  at  zero  frenuency  was  eliminated  from  the  computation  of 
the  mean  spectral  power.  This  resulted  In  considerably  more 
spectral  components  being  retained  In  the  smoothed  variable. 

Due  to  an  error  In  the  algorithm  the  variables  In  prototype 
analysis  were  smoothed  by  countlnn  the  total  number  (N)  of 
frequencies  with  spectral  value  greater  than  the  mean  spectral 
value  and  uslna  the  first  N freguencv  components  to  regenerate 
the  time  series,  Using  this  criterion  the  smoothed  data  was 
generated  utilizing  from  30  to  over  480  terms  of  the  Fourier 
series.  For  any  number  of  terms  selected,  the  resulting  curve 
represents  the  least  snuares  fit  to  the  original  data.  The 
least  souares  fit  Is  printed  and  plotted  In  "Matheny  FFT  1-22" 


for  each  of  the  22  variables  along  with  the  raw  data  on  the 
same  coordinate  axes.  The  Fourier  coefficients  used  to  re - 
generate  the  smoothed  data  are  listed  In  "f'atheny  FFT  1-22" 
and  appear  In  summary  In  "Mathenv  Meroe," 

5,8  The  identification  was  performed  on  the  smoothed  time  histories 
described  In  5.7,  The  pilot  was  identified  as  a linear  fixed 
order  system  with  17  Inputs  with  time  variable  coefficients. 

There  were  seventeen  Identification  Intervals  over  the  maneuver. 
The  A|  and  Bj  matrices  are  listed  in  "^atheny  ID  948,"  The 
number  of  Identification  intervals  is  determined  by  the  averaging 
criterion  Indicated  In  the  addendum. 

5.8.1  In  the  seventh  identification  Interval  the  singularity  of 
a matrix  which  requires  Inversion  in  the  identification 
aloorlthm  precluded  the  completion  of  Identification  for 
that  Interval. 

5.8.2  A cursory  examination  of  the  coefficients  of  the  A and  B 
matrices  revealed  that  the  coefficients  for  the  thirteenth 
Identification  Interval  were  oenerally  atypical  from  the 
coefficient  trend  from  the  beginning  of  the  maneuver  to 
the  end.  On  the  average  they  were  of  unusually  larqe 
maqnltude  In  comparison  to  neighboring  values.  The 
single  spurious  value  at  82  seconds  enters  Into  the 
computation  of  the  a and  b coefficients  In  the  thirteenth 
Interval.  The  effect  of  the  data  smoothing  procedure  Is 
to  Increase  the  effect  of  the  spurious  coefficient  on  the 
Identification  process  since  the  smoothing  procedure 
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affects  values  of  the  variables  approximately  one  second 
on  either  side  of  the  spurious  point.  The  signs  of 
spurious  points  do  not  appear  to  have  any  correlation  with 
the  normal  sign  relationship  of  the  data. 

5.9  In  order  to  present  the  A and  B matrix  coefficients  In  a summarized 
form,  a Fourier  series  was  fit  through  each  of  the  coefficients. 

A fast  Fourier  transform  routine  was  used  on  the  first  sixteen  co- 
efficient Intervals.  Since  the  seventh  Interval  coefficients 
were  undetermined,  the  seventh  Interval  coefficients  were  re- 
placed by  the  mean  values  of  the  sixth  and  eighth  Interval  co- 
efficients. In  order  to  compare  the  effect  of  the  thirteenth 
Interval  coefficients,  two  runs  were  made  one  with  the  thlrteenfh 
Interval  coefficients  as  computed  and  one  with  the  thirteenth 
Interval  coefficients  replaced  by  the  average  value  of  the  co- 
efficients of  the  twelfth  and  fourteenth  coefficients.  The  co- 
efficients of  the  Fourier  series  for  these  two  cases  did  not 
aopear  to  be  significantly  dissimilar.  No  statistical  tests  were 
made,  because  of  time  constraints  on  the  project,  Fourier  co- 
efficients for  the  Aj  and  Bj  matrices  are  listed  In  "h'athenv  FFT 
* AB  (thirteenth  coefficient  as  computed)  and  "Matheny  FFT  * AB2, 
(Thirteenth  coefficient  computed  by  averaging  the  12th  and  1 4th 
coeff ! dents) . 

6,0  Crltloue  of  the  methodology  and  suggestions  for  Improvement, 

6.1  The  presence  of  spurious  points  Is  definitely  detrimental 
to  the  method  of  analysis,  Oven  In  a maneuver  with  no 
spurious  values  apparent  from  examination  of  the  summary 
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listing,  spurious  values  appeared  In  all  22  variables.  In 
^ 20  of  the  variables  these  values  appeared  at  82  seconds, 

j The  magnitude  of  these  values  were  from  2 to  200  times  the 

i 

magnitude  of  the  normal  range  of  values  of  the  variables 
during  the  maneuver.  Usually  the  values  at  the  spurious 
points  were  ten  times  normal  magnitudes.  Two  of  the 
variables,  elevator  position  and  rolling  acceleration  had 
six  and  seven  spurious  points,  respectively.  Due  to  the 
Influence  of  these  spurious  values  on  the  Identification 
routine,  it  Is  necessary  to  remove  these  points  from  the  raw 
data  before  proceeding  with  the  analysis.  It  will  be 
necessary  to  examine  a listing  of  each  variable  or  to  devise 
a computer  routine  which  Identifies  values  which  change 
rapidly  In  .05  seconds. 

6.2  The  smoothing  of  the  raw  data  Is  necessary  to  the  Identification 
procedure  because  of  the  accuracy  and  resolution  of  the  re- 
cording Instruments  relative  to  the  range  of  the  variables 
recorded.  The  ranges  of  15  variables  are  less  than  ten  times 
the  accuracy  of  the  Instruments  used  to  measure  these  variables. 

For  5 of  these  15  variables  the  total  range  of  the  variables 

Is  less  than  or  eaual  to  the  accuracy  of  the  respective 
recording  Instrument,  For  the  elevator  the  total  ranoe  Is 
only  ten  times  the  resolution  of  the  recording  potentiometer. 
Consequently,  filtering  the  raw  data  to  remove  the  un- 
certainty due  to  Instruments  Is  necessary, 

6.3  The  raw  data  for  heading  contains  discontinuities  as  the 
headlnq  goes  through  0°  (360°),  It  Is  necessary  that  the 

i 

i 

i 

a 
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data  be  transformed  to  provide  a continuous  variable  from 


^ 0°  to  multiples  of  360°  before  processing. 

6.4  The  method  used  to  truncate  the  Fourier  series  should  be 
revised.  Visual  inspection  of  the  power  spectrum  of  each 
variable  and  estimation  therefrom  of  the  number  of  frequencies 
which  should  be  used  In  the  truncated  series  approximation  of 
the  variable  Indicated  that  significantly  fewer  frequency 
components  should  be  used.  In  some  cases  the  routine  used 
two  orders  of  magnitude  more  freouency  components  than 
estimated  thus  unnecessarily  extending  the  computational 
reoui rements. 

6.5  The  ptlot  Input  variable  (U|)  should  be  rewritten  so  that 
the  Identification  procedure  utilizes  the  true  variables 
upon  which  the  pilot  acts.  For  example.  In  the  30°  bank, 
the  pilot  manipulates  controls  In  response  to  the  difference 
between  the  actual  bank  angle  of  the  aircraft  and  the  desired 
hank  angle  (30°).  Similarly,  the  slonal  +<;  the  pilot  for  a 
constant  airspeed  criterion  is  the  difference  between  actual 
airspeed  and  desired  airspeed.  Some  of  the  pilot  state 
variables  which  are  deviations  about  a non-zero  position 
(e.g.,  elevator  position)  should  bo  rewritten  for  the 
identification  procedure  about  a zero  reference  position 
for  the  maneuver.  Some  of  the  apparent  sign  change  and 
time  variable  behavior  In  the  A|  and  B|  matrix  coefficients 
may  be  due  to  the  analysis  of  actual  values  rather  than 
error  values. 
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6.6  The  pilot  mav  not  be  adequately  described  by  a fifth  order 
system.  The  number  of  states  describing  the  pilot  may  be 
increased  by  auamentinq  the  state  vector  used  in  this  analysis 
with  the  derivatives  of  these  states.  In  order  to  estimate 
these  derivatives  the  smooth! no  of  the  variables  provided 

bv  the  power  spectral  density  method  described  In  5,6  and 

5,7  Is  essentia! , 

6.7  As  an  additional  comment  on  prospective  "ax  1 mum  Likelihood 
Parameter  Estimation,  In  view  of  the  fact  that  neither  the 
exact  functional  form  nor  the  number  of  states  of  the  pilot 
are  known,  none  of  the  statistical  conclusions  which  are 
Implicit  and  explicit  In  Its  structure  can  be  made, 


ADDENDUM  TO  APPENDIX  A 


1 .0  Purpose, 

The  purpose  of  this  addendum  Is  to  detail  the  rationale  for  analvsls 
of  the  T-37  Instrumented  aircraft  data, 

2.0  Summary. 

2.1  Because  of  the  nature  of  the  problem,  the  strateoy  adopted  was: 

2.1.1  analyze  the  power  spectral  density  of  each  measurement 
bv  utilization  of  the  Fast  Fourier  Transform. 

2.1.2  account  for  the  content  of  randomness  In  the  measured  data. 

2.1.3  accomplish  the  Identification  process. 

2.1.4  express  the  Identified  coefficients  as  finite  Fourier 
series. 

3.0  Rackoround. 

On  first  conslderlno  the  problem  It  was  proposed  that  the  standard 
state-space  based  methods  of  modern  control  theory  would  be  appropriate  to 
determine  the  eouatlons  which  would  predict  the  behavior  (as  reflected  by 
his  manipulation  of  the  controls)  of  a human  pilot  durlna  a specified 
maneuver. 

It  was  expected  to  be  necessary  to  use  either  the  form  (continuous  time) 
X (t)  = A (t)  X (t)  + B (t)  U (t) 
or  the  form  (discrete  time) 

£.1  + 1 ~ t\  I + Bl  !^l 

It  was  thounht  possible  that  theXj'swould  be  the  aircraft  controls' 
positions  and  the  Uj’s  would  be  the  measured  cues  which  help  a pilot  make 
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his  decisions.  It  was  recognized,  however,  that  the  pilot  might  have 
more  sionificant  states  than  the  number  of  aircraft  controls  and  that 
there  ^ioht  he  other  sionificant  Innuts  (besides  measured  cues).  In 
particular,  snurious  nilot  and  measurinn  system  behavior  was  an  item  of 
concern  as  was  rate  of  climb  and  pilot  "dither". 

it  was  decided  to  proceed  in  such  a way  as  +o  minimize  cost  of  the 
nroject  which  involved  formulatino  the  following  hypotheses: 

(a)  the  discrete  time  form  with  an  uncertainty  vector  Is  adeouate 


X =•  A X + 0 

-i  + i ' -I  i 


u.  + n, 

”i  Ji 


(b)  the  aircraft  controls'  positions  are  the  states,  and  measured 
cues  are  the  inputs, 

(c)  the  coefficients,  A(  and  B.  are  slowlv  tine  varying 

* I Ai  + 1 00/  AP  “• 1 & lBi  + l00/  BJ  = -l)* 

(d)  the  uncertainty,  g,  , is  a random  seouence  with  zero  mean. 

(o)  the  measurement  of  the  state,  VXj , differs  from  X ^ by  random 

variable  seouence  with  zero  mean  , and  the  measurement, 
vu ■ , has  a similar  property. 

Tiie  above  hypotheses  oivel 

+ y x,  - o>y ' 


and 


Yu  wu 

“i  “l 


YX  - uX 

l + l — i + | 


= A 


. fe  ' <1 


+ H.  pYU  - WU 

' L—  -'J 


+ n 
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Averaainp  over  l adjacent  periods  would  yield 


* 

YX 

- uX 

, * 

Ai 

YX 

- wX 
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9. 

Given  the  hypotheses,  If  i were  lame  enouch,  the  noise  terms  would  be 
averaoed  to  zero: 


= A! 

YX 

+ r4. 

i 

vu 

l 

_ - 1 

9 

i. 

l 

and  this  forns  the  basis  for  our  identification  tratenv, 

T fie  results  of  nrel Iminary  analysis  indicate  that  Aj  and  Rj  are  not  slowly 
vary i no.  The  reason  could  be  that  there  are  more  pilot  states  than  air- 
craft controls,  or  the  effect  of  larpe  scale  randomness  cannot  be  smoothed 
by  averaoino  over  a few  adjacent  points.  A third  possibility  Is  that  other 
pilot  cues  than  the  measured  ones  are  slnnlf  leant.  Rate  of  climb  Is  an 
examp  I e , 

All  these  possibilities  renulred  consideration  of  more  expensive 
approaches  to  the  problem.  Some  possibilities  were: 

(a)  An  Iterative  maximum  likelihood  identification.  This  approach 
was  deduced  to  be  far  too  expensive, 

<h)  Remove  the  randomness  (as  far  as  is  statistically  defendabie) 

from  the  measurements  and  then  proceed  with  the  Identification, 

This  procedure  was  much  less  costlv  and  was  projected  to  nlve 
rood  results. 
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4.0 


Analysis  Procedure  Adopted 

4,1  f'ake  the  fol lowinn  hypotheses, 

(a)  the  random  component  Is  a stationary  process.  The  probabllit 
density  function  has  an  unspecified  distribution  (l.e,,  It 
nay  be  unicorn,  Gauss  Ion,  etc,).  The  conditional  pro- 
babilities hence  the  n dimensional  probability  density 

is  such  that  power  distribution  over  frecuency  is  flat 

< < 


up  to  a finite  freruency,  f.,  such  that  fg  - f 
where 


t 

T' 


y 


f 


T = 


maneuver  duration 


f = maximum  freauency  content  of  the 

hypothesized  deterministic  slcnal 

4.2  I'slnq  the  Fast  Fourier  Transform,  obtain  the  finite  Fourier 
Series  which  fits  the  2048  data  points  for  each  measurement, 

4.3  Determine  t'-e  power  spectral  density  (PSD)  of  each  measurement, 

4.4  Deduce  randomness  components  from  the  htqh  frenuency  components’ 

f I afness. 

4.5  Subtract  the  (PSD)  of  the  randomness  component  from  the  overal 1 
PSD  thus  vleldlno  a PSD  of  the  deterministic  component,  This 
is  done  for  each  measurement,  (Preliminary  studies  show  that 
the  detern i n . st I c component  Is  contained  In  the  first  (lov/est) 

3 f>  of  the  Fourier  spectrum).  This  also  establishes  the  hlohest 
froouencv  of  the  Fourier  Series  of  the  determl n I st I c component, 

4 ,(>  Use  the  Fourier  Series  so  derived  to  ne nerate  data  for  the 
i sent i f i cation. 
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4.7  In  the  Interest  of  economy,  use  the  same  1,0,  program  we  have 

usea:  Generate  coefficients  for  each  six  (6)  seconds, 

4.8  Fit  a 20  - term  Fourier  Series  to  the  generated  coefficients, 
5,0  discussion 

Hy  ta^inn  this  approach,  states  could  be  added  by  includinn  their 
derivatives,  and  derivatives  of  Innut  can  also  be  considered. 
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